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ABSTRACT 

This research analyzes two approaches to the economic evaluation of an aircraft 
modernization program. The Analytic Science Corporation (TASC) method is compared 
with the Logical Decision for Window (LDW) methodology. TASCFORM-AIR model 
is a method to quantitatively measure military force modernization. Logical Decisions for 
Windows software and methodology is based on Multiattribute Utility Theory. It also 
helps to evaluate decisions quantitatively. 

The research includes analysis of the reasons, constraints and tendencies in the 
modem aircraft modernization process. Weapon modernization is usually driven by 
several objectives, all of them in one way or an another are pertinent to resource 
allocation. Reliable analytical tools are important to make good decision. Cost- 
effectiveness and cost utility approaches are evaluated. 

Comparison of both methodologies is based on the MiG-29 modernization 
situational model. TASCFORM-AIR Model provides static indicators of military force 
potential. This can be viewed as measures of effectiveness. The LDW program computes 
the alternatives’ utility by combining its measure levels based on the analyst’s 
preferences. The results produced in both cases are useful in several ways. They are 
indicators, however indicators rather than "answers" to the decision making problem 
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I. INTRODUCTION 



A. BACKGROUND 

There is no need to be an expert in aviation to notice a significant drop in the 
production and design of new combat aircraft at the present time in all leading countries. 
Among the main reasons for this, one can mention the end of the Cold War and the 
collapse of the Soviet Union; others will emphasize economic difficulties and the rapidly 
increasing costs for new aircraft design and production. The countries making new 
weapon platforms produce quantities of aircraft much less than the levels of the 70s and 
80s. 

The days when the United States and the Soviet Union produced a new class of 
combat aircraft every ten years and a new transport every fifteen years are gone. 
Procurement of new military aircraft now is more a political and economic issue than 
purely military. In addition, the life cycle cost of the whole system plays as important a 
role in source selection as combat characteristics. 

Nevertheless all military doctrines and recent combat experience show the 
increasing role of aviation in the structure of military forces. For the military economist, 
it means that to keep old aircraft safe and able to perform modem operational tasks 
require some additional money. In those conditions many countries have chosen the 
upgrade and modernization path. Many studies show that most jet combat aircraft of the 
so-called “third generation” have great potential for modernization and upgrade using 
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modem technology. Even aircraft veterans, such as the MiG-21 and the F-5, will get new 
life, and the description of the modernization programs for MiG-29 and F-15 have 
appeared in all specialized magazines. 

In many cases, industry, trying to find new markets, proposes its own programs to 
customers. But generally, aircraft upgrade programs tend to be customer driven, with 
industry responding with a solution tailored to specific customer requirements. On the 
other hand, a number of pitfalls exist on this path. Two basic conditions have vital 
importance for any aircraft upgrade plans. First, the basic airframe must have an 
acceptable future service life and, second, ensuring an adequate holding of basic spare 
parts is important. 

The aircraft built before 1970 had service lives of 15-20 years. Later, the 
electronic component of the aircraft became increasingly important, and in recent models 
designers have pre-planned the mid-life upgrade. Today, the cost of the frame and the 
engines are probably less than 50% of the total aircraft cost, and aircraft such as 
“Eurofighter” have built-in upgradeability from the start that allows the full potential of 
the upgrading to be realized. 

Clearly, upgrading and modernizing are only a partial solution. Future needs will 
demand replacing old machines with new ones. Even the best modernization program is 
not a panacea for new procurement, but modernization can buy the user time to consider 
the alternatives. Modernization can be a direct replacement by buying a new generation 
of the same class of aircraft, or it can be indirect, by giving an aircraft a new role. 
Modernization “ can reduce cost-of-ownership for the later part of an extended sendee 
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life, and it can offer interim capability until the next generation comes along or an 
improvement is made in an existing role. The decision to upgrade lies with the user 
alone.” (Gething p. 2) 

The never-ending researches of designers and producers of the aircraft, engines, 
equipment and armament increase the number of alternatives available. What 
combination to chose? Which characteristics are more important? Having constrained 
budgets, military planners often face complicated decision problems with great 
uncertainties and multiple inputs and outputs. Good intuition and rich experience are not 
sufficient to reach the best solution. Modem economic and military sciences offer a great 
number of analytical instruments for decision making and support. Which one best fits a 
modernization decision? 

In my thesis I will evaluate widely used scientific analytical methodologies as to 
their practical use by military managers for modernization decision-making. 

B. PURPOSE 

By considering the vital role of research tools for effective allocation of budgetary 
recourses, I will evaluate selected methods, approaches, and real programs for decision- 
making in aircraft upgrade and modernization. 

The main purpose of this thesis is to estimate data, examine assumptions and 
evaluate the results of different methods and to make recommendations for their practical 
use within the Armament Department of the Ukrainian Ministry of Defense. The case of 
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the MiG-29 modernization will be used to evaluate research tools and to illustrate 



recommended methodology. 



C. RESEARCH QUESTIONS 



The research questions due to the nature of the problem, and because of the use of 
situational modeling for methodology evaluation are divided into two groups. 

Group I. Pre-modeling questions: 

• What are the main reasons for combat aircraft modernization and what are 
the objectives and the constraints on modernization? 

• What generalization and assumptions can be made in modernization 
modeling? 

• How can cost-effectiveness and cost-utility analysis be used for 
modernization decision-making? 

• What specialized and universal decision support programs and software 
exist? 

Group II. Post-modeling questions: 

• What criteria can be used for methodology selection? 

• What are the key dimensions of the methodology selection problem? 

• What are the main measurement-scale selection and construction issues 
relevant to given multi-attribute-utility analysis? 

• What are the advantages and disadvantages of assessed methodologies? 
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II. AIRCRAFT MODERNIZATION DECISION 



The improvement or replacement of military weapons system is generally 
referred to as weapon modernization. The ostensible purpose of such 
modernization is to correct shortcomings in the original weapons, or 
respond to either changes in the weapons purpose or to changes in enemy 
weapons. ( Sorenson , p. 23) 

There are three ways to modernize existing weapons. Modem aircraft as a system 
can be improved: 

• as a platform (speed, maneuverability, range). 

• as a part of strike complex (command control, communication and 
navigation). 

• as a weapon (power, accuracy, effectiveness). 

Modernization may change just a single element of a weapon system or it can be a 
part of state weapon program. For example, the latest modification of the F-16 after a 
midlife upgrade is believed to be two or three times more effective that its predecessor. 
(Sweetman, p. 26) 

Technologically speaking, modernization can be as evolutionary as the case of the 
MiG-21 that was gradually transformed many times during its 45-year history, or as 
revolutionary as when one technology is completely replaced by another. For example, 
the transformation from piston to jet engines in aviation was clearly revolutionary. 

Technology can influence not only changes in weapons, but can also transform 
war itself. In this Chapter, I will evaluate modernization as a subject of policy that can 
impact the defense resource allocation. 
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A. 



REASONS AND OBJECTIVES 



Weapon modernization is usually driven by several objectives, all of them in one 
or other way are pertinent to resource allocation. Modernization can result from national 
security requirements or particular service demands based on strategy development. In a 
political context, it can arise from regional or group interests. Analyzing military 
aviation development we can define how the following typologies explain weapon 
modernization: 

• Modernization due to strategic necessity. National security defines threats 
and then strategy produces weapon performance demands; 

• Weapon modernization based on demand to match a new military 
capability of a rival nation; 

• Modernization based on interest of powerful domestic groups that 
benefited from military founding; 

• Weapon modernization as a means to maintain a defense industrial base; 

• Modernization opportunities based on scientific and technological 
innovations; 

• Modernization as the result of shifted control over the military acquisition 
process. 

We can separate these reasons on two levels: strategic (the first two) and 
structural (the others). The first group is connected to the international environment as 
the stimulus for weapon development; the second group focuses on domestic policies. 

To better understand the modernization decision making process, I’ll evaluate 
separately the influence of state strategy and military competition, the role of military 
doctrine and the place of modernization within the military planning. To make this 
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analysis more relevant to methodology selection, I will evaluate the technical level of 
modernization caused by flight safety, effectiveness and obsolescence. Even though 
technical problems are partly included in strategic and structural levels, evaluating them 
separately is reasonable in the context of this thesis. 

1. Strategy and Doctrine 

Strategy connotes the linkage between national political objectives through the 
threat or use of the military forces. (Sorenson, p. 26) For example, for the United States, 
the Cold War strategy was a combination of deterrence and containment. Both were 
based on the expectation of expanding Soviet power and on the belief that military force 
was an appropriate policy instrument for containing Soviet expansion. One doctrine that 
was selected to support these strategic objectives was the employment of Air Power. 
However, the strategic environment has changed. William Cohen in his Testimony 
before the Senate Armed Services Committee on February 5, 1998 said, “ The future 
conflict environment will present challenges that are dramatically different from those 
confronted by our armed forces during the Cold War and even the Persian Gulf War. 
Consequently our current concept of tactical air forces will likely have to change 
dramatically.” 

Presently, the strategy involves minimizing serial production runs of new aircraft 
under all but two circumstances: first, when the new system offers a major boost in 
military effectiveness that solves a major strategic or operational problem, and second, 
when existing systems have reached the end of their useful lives. 
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Doctrine refers to the operating principles of an organization that link weapon 
application to a specific end. Doctrine translates the generalities of strategy to operating 
principles. For example the Air Force Modernization Planning Process (AFMPP) is part 
of the Air Power doctrine and the mechanism for supporting the five core competencies- 
air superiority, space superiority, precision employment, global mobility and information 
dominance. The AFMPP integrates the elements that provide the foundation for the five 
competencies into a coherent modernization plan that reaches 25 years into the future. 
The foundation elements included in the modernization plan are: 

• readiness and sustainment; 

• research, development, test and evaluation; 

• logistics; 

• information technology; 

• equipment and facilities; 

• manpower. 

However, replacement of aircraft has slowed significantly in recent years because 
of budget constraints and affordability. The result has been a shift to increased upgrading 
and life extension of many systems beyond what was typically done in the past. 
(Butowski, p. 3) 

In an ideal world, strategy and doctrine ought to provide directions for persons 
who are in charge of design and procurement of weapon. But strategy and doctrine are 
abstractions, which do not translate easily into the precise requirements for weapons. So, 
there are other reasons that can explain aircraft modernization choice. 



8 



2 . 



Readiness and Effectiveness 



One of the earliest paradigms in the literature of social science is the arms race 
typology. This view to modernization was originally devised by Lewis Fry Richardson 
who developed models of arm acquisition with the hope that understanding the process 
might help in preventing war. (Sorenson, p. 39) The fundamental assumption behind 
arms race models is that two hostile nations will arm themselves completely, with one 
nation stimulating the arming of the second in a reciprocal fashion. Arm races are thus 
“action-reaction behaviors,” with each nation trying either to win the arms race at best or 
at least is trying to avoid losing it. Nations as a result, arm themselves for external 
reasons, reacting to a perceived threat. 

Strategic and structural categories of arms races influence the technical level of 
the modernization process, through the demands of military readiness and individual or 
group effectiveness. Joint Pub 1-02 defines readiness as “the ability of forces, units, 
weapon systems or equipment to deliver the outputs for which they were designed 
(includes the ability to deploy and employ without unacceptable delays).” 

As technical element of modernization, readiness and effectiveness have time, 
range, loading and accuracy components. For example Russia’s program of tactical 
aviation development will include the following directions: prolongation of the service 
time, modernization and improvement of the currently used fighters and missile 
armament to broaden their ability to destroy both the aerial and ground targets, and 
improving of range and autonomy. The next direction is standardization of the fighter 
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aviation units training within the framework of Air Force and Air Defense formations to 
provide both air defense and battlefield support. (Butowski, p. 7) 

The essential aims of Russia’s Air Force modernization are as follows: 

• an increase in combat effectiveness, practically due to the application of 
new weapons previously prepared for the next generation aircraft; 

• a multi-role capability in combat aircraft obtained through the use of new 
fire control and weapon systems; 

• an enhanced night and adverse weather mission capability; 

• an expansion of the “information field” for aircrew through the use of new 
navigation systems, data links, etc; and 

• a reduction in maintenance cost during the prolonged service life of 
aircraft by replacing standard maintenance schemes with maintenance 
according to the actual technical conditions of aircraft. 

The results of modernization in Russia are characterized by the growth of the 
integral parameters, such as the coefficient of the combat potential while attacking the 
aerial or ground targets. The Mikoyan design bureau representatives presenting MiG- 
29SMT program tell that modernized MiG-29 combat efficiency in an air-to-air role will 
be 2.1 : 1 compared with the current MiG-29 ( and even 6.5 : 1 in long distance air 
combat). The combat potential of the aircraft in air to ground mission will supposedly be 
increased by a factor of 3.8 , or even 8 against point targets. (Polushkin, p. 6) 

3. Flight Safety 

Flight safety has a special position among the reasons and objectives of 
modernization. It is only one indisputable reason because of the value of the pilot’s life, 
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and threat of losing highly expensive weapons due to technical problems make the 
military and the producers cooperate to improve safety and reliability. 

When flight safety design flaws are discovered, the consequence is usually the 
suspension of operations. Designers and producers use all their resources to eliminate or 
to neutralize possible threats. If flight restrictions don’t correct a flight hazard, the 
design bureau has to make structural changes in the shortest possible time. This has been 
labeled the so-called “urgent modernization.” 

Another level of modernization relevant to flight safety is pre-planned or 
preventive upgrade. Employment of aircraft reveals many weak or problematic areas. 
Sometime they are not urgent or occur infrequently. Analyzing defects and malfunctions 
at the aircraft design bureaus creates upgrade programs for regimental and depot levels of 
maintenance. The regimental specialists usually change specific details and elements, 
while depot personnel traditionally make structural changes. Producer specialists or even 
design engineers can make highly complex upgrade and modifications. 

Soviet era statistics show that for tactical aircraft, such as the MiG-25PD, the 
number of elements to be upgraded would generally total 10 to 15 after each 1500 flight 
hours. At the same time, from 1980 to 1988 flights of MiG-25’s were suspended six 
times and flights were restricted until structural changes were made more than 15 times. 
The cost of a safety related modernization might represent from 8 to 10 % of the lifetime 
aircraft maintenance cost. 
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4. 



Obsolescence 



Examples of modernization mainly caused by obsolescence are the Soviet MiG- 
21 and the American F-5 upgrade programs. These also are examples of modernization 
that were created to satisfy customer needs. Countries, such as Romania, Egypt, Turkey 
and Taiwan, which possess these aircraft do not have sufficient funds to buy new or even 
used aircraft and have to use available airframes. In both cases, design bureaus agreed 
that these aircraft are still in good condition and their useful lifetime can be prolonged for 
10 to 15 more years. Another reason these countries can employ these used aircraft is 
that the air industry has created a lot of universal systems and elements that can be easily 
adapted to any airplane. Examples are the Global Position System (GPS) navigation, 
radar and universal weapon delivery systems. On the other hand air forces of many 
countries have numerous special tasks (patrolling, training, counter terrorists operations 
and so on) that allow using aircraft with comparably low performance. So, the primary 
problem is the obsolescence of existing armament and equipment. From material 
published in aviation journals, we know that in both cases (MiG-21 and F-5) objects of 
modernization are cockpit equipment, navigation, communication, radar and armament. 

The advantage of such modernization is a high level of cost effectiveness index. 
For example the cost of the MiG-2 1-98 upgrade program is about 30% of an aircraft’s 
initial cost in constant rubles, and the estimated increase of effectiveness is from 1.7 to 
2.5. The problem is that the technological and information revolution dramatically 
increases “the depreciation rate” of new equipment and weapon systems and as a result 
the comparative effectiveness of old aircraft may be worse than predicted. 
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B. CONSTRAINTS 



1. Technical and Technological 

There are two fundamental requirements to be fulfilled before any aircraft can be 
upgraded or new equipment can be installed. First and foremost, the basic airframe must 
be sound with an acceptable service life remaining. The airframe structure can deteriorate 
depending on the nature of its use and the level of maintenance it has received. 

Second, if the airframe is in good condition and well maintained, before 
proceeding with the upgrade, it is important to ensure an adequate availability of basic 
spare parts. If the storage of spare parts is sparse, it is necessary to estimate the 
possibility of ordering parts from the original or side producer. Both options could be 
expensive and could significantly increase maintenance cost. 

One more important issue at the present time is the movement from analog to 
digital electronic systems. For old aircraft, a partial upgrade can create two incompatible 
electronic systems. 

The next problem is adapting new weapons to the airframe. Sometimes the 
weight of additional frame elements to adjust to the new weapon to an old aircraft may be 
higher than the weight of the additional weapon itself. This increase in weight might lead 
to another problem: insufficient power. Of course, experience also suggests that many 
systems could be lighter than the ones they replace. However, aircraft generally gain 
weight during upgrades, even if this is in the form of additional weapons-carrying 
capability. Obviously, ascertaining whether the engine(s) will have sufficient power to 
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deliver an equal or better thrust-to-weight ratio in the upgraded aircraft is important. 
(Gething, p. 5.) 

There are several arguments concerning how technology influences 
modernization. On the one hand strategy and doctrine may guide the technology of 
weapon development, stimulating technology. On the other hand, it has also been argued 
that technology itself simulates and constrains the development of strategy, since 
technological development can make new strategies possible. Yet another argument is 
that technology reduces uncertainty and its use may be maximized without specific 
knowledge about the present or future state of adversary capability. New technology is 
sometimes developed independently from weapon projects, but then incorporated into 
them because it is available. 

2. Finance 

In the year 2054, the entire defense budget will purchase just one aircraft. 

This aircraft will have to be shared by the Air Force and the Navy 3.5 

days each per week except for leap year, when it will be made available to 

the Marines for the extra day. 

Norman R. Augustine 

As economic difficulties impact a world already wrestling with reduced defense 
budgets, the hard fact is that fewer new weapons platforms are being developed. Yet it is 
not only the budgetary impact that is decreasing this development. The state’s military 
budget expands all security problems including strategy and force structure. 

Balancing fiscal constraints against mission requirements is a relatively new 
problem for military planners in the former Soviet republics. New democratic processes 
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dictate that the military compete for funding along with the other needs of the nation. In 
the 1980s the Soviet Union spent about 20% of the GNP on the military, while the 
Ukraine presently can spend only 2% of the GNP. Money available for military 
programs is significantly different from that ones available ten years ago, and so are 
possibilities of military industry. We can expect this state of affairs to continue in the 
future. 

Even so, after the Gulf War and the Serbia conflict the world was convinced of 
the importance of air power in the modem conflicts; however, any nation can spend only 
a reasonable amount of money for defense. The age-old question of “How much is 
enough?” will surely continue. 

As we look at the cost of modernization, high technology has an attractive lure. 
Technology offers high military efficiency, stability and economy. But at some point in 
the development of a weapon system, the additional cost of increased capability 
outweighs the gains. 

3. Scientific and Industrial Base 

When analyzing particular aircraft modernization programs, it is easy to say that 
the main role in definition of such a program belongs to the original design bureau and 
manufacturer. Two scenarios are possible. First is the modernization of domestic aircraft, 
when all stages of an aircraft lifecycle belong to country-owner. This is the simplest case 
and this modernization is usually constrained only by a lack of money or technology. 
Second is when the owner bought or inherited aircraft without previously agreed 
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modernization. In this situation, modernization is also strongly constrained by the 
relations with the designer and manufacturer. 

Even if the country has a modem scientific and industrial base, it can’t modernize 
significantly without cooperating with the design bureau. The design bureau owns the 
technical information concerning the particular aircraft, and foreign customers might 
select only those programs proposed by a designer, or they might order a new project. 
Such an upgrade program may require much more money and time. 

The design bureau is best qualified to estimate the lifetime for the frame and 
elements. It studies possibilities to change systems and armament and finally produces 
an upgrade program. Examples of the MiG-21 and MiG-29 modernization show that the 
initial intentions of some nations, even those with modem air industry, to undertake their 
own modernization finally led to a close cooperation with the original designer. The 
main reason for the design bureau’s leadership is that it possesses full technological and 
construction documentation. Any changes made by another firm may weaken the 
design or introduce incompatible elements in operation. If uncoordinated changes are 
made, the design bureau can deny any responsibility for future flight safety. 

Another constraining factor for countries that cannot produce their own aircraft is 
an insufficient industrial base for providing an upgrade. For example, all Warsaw pact 
countries inherited air depots able to provide high quality maintenance and repairs. 
Depending upon the level of design changes, modernization can be made by a depot (as 
in the case of the MAPO- DASSO program for MiG-29) or by a main plant (as in the 
case of the MiG-29SMT modernization). Clearly, modernization conducted by a 
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domestic depot is cheaper and gives additional advantages, such as professional repair 
personnel and the relative independence from manufacturer in future aircraft 
maintenance. 

C. DECISION MAKING METHODOLOGIES 

What makes decisions hard? Certainly different programs may involve unique 
difficulties. For example, any modernization decision requires us to think about the 
interests of various groups, as well as to consider the limitations of information on the 
inputs and outputs. Although every decision may have its own special problems, there 
are four basic sources of difficulty. (Clemen, p. 2) First, a decision can be hard simply 
because of its complexity. In case of modernization, an Air Forces must consider many 
individual issues: prices, effectiveness, lifetime, industry base, etc. Second, a decision 
can be difficult because of the inherent uncertainty in the situation. Third, a decision- 
maker may be interested in working toward multiple objectives, even though progress in 
one direction may impede progress in others. Finally, a problem may be difficult if 
different perspectives lead to different conclusions. Or, even from a single prospective, a 
slight change in certain inputs may lead to different choices. Different individuals may 
look at the problem from different prospective, or they may disagree on the uncertainty or 
value of the different outcomes. 

So, which methodologies exist to help us make a decision? They are legion. 
Different schools, approaches and theories give us a large selection of scientific methods 
for decision making and support. The military planners use four main groups of 
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methods: benefit-cost, cost-effectiveness, cost-utility analysis and dynamic modeling. 
Let’s examine the main features of these four groups. 

1. Benefit-Cost Analysis 

Benefit-cost analysis is recommended by the US Office of Management and 
Budget (OMB) as the technique to use in a formal economic analysis of Government 
programs or projects. According to the OMB Circular A-94 “Guidelines and Discount 
rates for Benefit-Cost Analysis of Federal Programs,” benefit-cost analysis (BCA) is used 
“to support governmental decision to initiate, renew, or expand programs or projects 
which would result in a series of measurable benefits or cost extending for three or more 
years into the future.” 

The standard criterion used in BCA is the net present value (NPV) the discounted 
monetized value of expected net benefit. The NPV is computed by assigning monetary 
values to benefits and costs, discounting future benefits and costs using an appropriate 
discount rate, and subtracting the sum total of discounted cost from the sum total of the 
discounted benefit. Discounting benefits and costs transforms gains and loses occurring 
at different times to a common unit of measurement. Circular A-94 states that programs 
with a positive NPV increase social resources and are generally preferred. Programs with 
a negative NPV should generally be avoided. 

The main problem of this method is that the NPV is not always computable 
because monetary values of some benefits and costs cannot be determined. To overcome 
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uncertainties in the BCA Circular 94A proposes to use comprehensive enumeration and 
quantifying of benefits and costs. 

The main elements of the BCA are: 

• Policy rationale. A rationale should be clearly stated in the analysis; 

• Explicit assumptions. The analysis should include a statement of the 
assumptions, the rationale behind them, and the review of their strengths 
and weaknesses; 

• Evaluation of alternatives by means, by different program scales, by 
methods of provision and by different degrees of government 
involvement; 

• Verification of determination whether all anticipated benefits and costs 
have been realized. 

Those working with the BCA should consider such variables as incremental 
benefits and costs, interactive and international effects, inframarginal and indirect 
benefits and costs. After that, analysts should include calculations for inflation and risks. - 

The results from the BCA are explicit and obvious. However, difficulties with the 
definition of benefits and costs of military indexes relevant to combat effectiveness make 
the BCA rarely used in military planning. Even though the BCA has special tools to deal 
with uncertainties, the huge number of those uncertainties makes it extremely difficult to 
use in real tasks. 

Three limitations of the BCA are common. First, analysts may be unwilling or 
unable to monetize the most important policy impacts. Second is that any particular 
effectiveness measure does not capture all of the social benefits of each alternative. 
Finally, analysts frequently deal with intermediate goods which linkage to preferences is 
not clear. 
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In such cases, commonly-used alternatives to the BCA are cost effectiveness and 
cost-utility analysis. 

2. Cost-Effectiveness Analysis 

If all alternatives are mutually exclusive, and the status quo is among the 

alternatives, sharing similar scales and patterns of cost and benefit, then 

cost effectiveness analysis does select the most effective policy. 

(Boardman, p. 396) 

OMB opinion holds that cost-effectiveness analysis (CEA) is less than a 
“comprehensive technique, but it can be appropriate when the benefits from competing 
alternatives are the same or where a policy decision has been made that the benefits must 
be provided.” 

According to the OMB Circular NO A-94 a program is cost effective, if on the 
basis of life cycle cost analysis of competing alternatives, the program is determined to 
have the lowest cost expressed in present value terms for a given amount of benefits. 
Cost effectiveness analysis is appropriate whenever it is unnecessary or impractical to 
consider the dollar value of the benefits provided by the alternatives under consideration. 
This is the case whenever (1) each alternative has the same annual benefits expressed in 
monetary terms; or (2) each alternative has the same annual effects, but a dollar value 
cannot be assigned to their benefits. An analysis of alternative defense systems often 
falls in this category. 

In comparison with the BCA, cost-effectiveness analysis produces a ranking but 
does not provide explicit information about whether there would be positive net social 
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benefits associated with any of the alternatives being considered. Since CEA does not 
monetize benefits, it inevitably involves two different metrics: monetary cost and an 
effectiveness measure. Because of the nature of these metrics and scale differences one 
cannot add or subtract one from another. Thus it is a measure of economic efficiency. 

In practice, CEA almost inevitably reduces negative impacts that would be 
included in BCA. Indeed, CEA typically considers only the measures of effectiveness. 
Military projects usually have multiple objectives and benefits. Moreover, the necessity 
of obtaining an arbitrary ranking increases, as alternatives become less similar in terms of 
the inputs and outputs. From this point we enter the area where cost-utility and multi- 
attribute utility analysis give the best result. 

3. Cost-Utility Analysis 

Cost-utility analysis (CUA) relates budgetary cost to a single benefit measure, but 
its benefit measure is a construct composed of several benefit categories. Boardman 
believes that “CUA can be thought of as a form of CEA employing a more complex 
effectiveness measure.” (p. 403) As CUA involves two or more distinct variables the 
analyst must designate how these variables are to be defined and combined. This is a 
problem in multi-attribute decision making. 

Utility analyses are useful in two important areas: (1) to quantify attributes that 
don’t have an obvious unit of measurement, and (2) to quantify and model a decision- 
maker’s propensity to accept or to avoid risk. (Marshal, p. 249) The most difficult part 
of such an analysis is to obtain a utility function. Sometimes it is unrealistic, unclear or 
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too risky. But this does not negate the usefulness of this procedure in decision modeling. 
Indeed, the process of utility function construction often gives insight and clarity to the 
problem. 

There are two ways to determine a utility function when decisions are made under 
conditions of uncertainty. One is when an indifference probability is found for a fixed 
risky venture. Another way to determine a utility function is when a riskless alternative 
payoff is found that is equivalent to the expected payoff of a given risky venture with 
even odds of success. 

At a strategic level of military planning, the effectiveness measure for a decision 

problem may be captured by criteria such as readiness, sustainability or force projection. 

Some attempts to model these problems have an inherent but usually unstated, 

assumption that because no measurement units exist, the attributes can be represented by 

dimensionless quantities. To prevent inconsistencies in model output, Kneale Marshal 

states and uses the following principle as a fundamental guide in model building: 

For a multi-attribute decision model to be consistent it should apply the 
same rules for combining attributes that cannot be measured directly as it 
does for those that can. If the problem under consideration has 
performance attributes for which there are no obvious measurement units, 
one should not assume that the weights assigned to these attributes are 
dimensionless and hence can be normalized in an arbitrary manner. 

4. Dynamic Modeling 

Computers allow analyst to make huge calculations rapidly. Computers also allow 
them to use more and more complicated models in simulation of real processes. There 
are many different models for simulation of military actions and even regional wars. 
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Many military planners and scientists have found these models useful for resource 
allocation planning. Multi-level dynamic models can simulate reality, changing inputs in 
accordance with a given scenario and state of outputs. Modem models of global and 
local military conflicts are based on complicated probabilistic scenarios and complex 
mathematical theories. These models let researchers evaluate the influence of entire 
weapon systems, or just their parameters, on the result of simulated military operations. 
Clearly the quality of the results in dynamic modeling depends strongly on the details of 
the model, the quality of assumptions and simplification. The results received in 
different stages of simulation may then be used to construct specific cost-effectiveness or 
cost-utility indexes for decision making. 

For example the DynaRank methodology (Hellestad, p. 7) can work with multiple 
objectives and rank policy options by cost-effectiveness. Each ranking is a judgement 
about the relative importance of higher level objectives and a variety of success criteria. 
It can also be used to integrate a detailed analysis with emphasis on components of 
defense strategy. 

On the other hand, complicated models have to work with stochastic elements 
involved in the simulations. These means that every new run can give different results, 
and repeated simulation runs may result in different sets of summary performance 
measures. 
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B. ANALYSIS OF ALTERNATIVE MEASUREMENT SCALES 



Decision-makers deal with four different levels of measurement: nominal, ordinal, 
interval and ratio. Money, often the main index in analysis, has ratio measurement. 
Analysts always try to work with compatible indexes when assessing different policy 
alternatives. In this case it would be easy to say that doubling commodities doubles 
money cost. Unfortunately, when we use effectiveness or utility as indexes in analysis, it 
is sometimes very difficult to make the right decision due to the scale differences. 

In simple cases, when all of the policy alternatives have the same cost, scale 
differences do not cause a problem. If, in addition, the cost-effectiveness analysis is 
inclusive of all social costs and benefits, then CEA ranks alternatives in terms of 
allocative efficiency. Similarly, scale is not a problem if the level of effectiveness is 
constant across all alternatives. In case of fixed effectiveness, CEA corresponds to a 
simple cost minimization problem, while in the fixed budget case CEA corresponds to a 
simple effectiveness optimization problem. 

In real problems we can find that large scale differences among alternatives can 
distort choice. For example, we have two exclusive alternatives. In the first case we can 
invest $10 million in flight safety and save ten lives a year. The second alternative will 
cost $200 million and would be able to prevent 100 military casualties in a conflict. Due 
to the simple cost effectiveness ratio, the first alternative is twice as effective as the 
second. But the second alternative can save ten times more lives in certain conditions. 

A different problem arises if the decision-maker must select between indexes that 
are in different measurement scales. An example of such a problem is how to select 
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between the ordinal ranking of some radar qualities and the ratio measures of additional 
weight relevant to this change. Different methodologies propose different approaches to 
convert scale and measurement differences to compatible results. 

The first method is normalization. TASCFORM methodology, for example, uses 
the F-4B as a basis for comparing different aircraft. Then it computes the ratios of all 
performances of evaluated aircraft to F-4B’s performances. The methodology effectively 
constructs a ratio scale in terms of F-4B equivalents, but the constructing scoring system 
doesn’t determine the utility of F-4B equivalents. 

Utility theory proposes another way to solve a measurement problem. Single 
utility functions (SUF) let us transform differently measured attributes to a common unit 
called “utility.” Actually SUFs play the role of normalization of different measures to 
common measures with interval values. Individual trade-off between attributes, in 
different methodologies that use utility theory, are usually based on personal judgement. 
In this case we receive an interval ranking of alternatives. 

Probabilities and uncertainties when added to analysis may change scales of 
results. As a result, when the lottery option discussed above is used, one can obtain a 
scale that has interval properties in the sense that linear transformations of the utility 
function can be taken without changing the decision. However, one really obtains an 
ordinal scale for utility when the developed utility function is used to assess alternative 
combination of utility and some other variable, such as cost, is not included in the utility 
function. 
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III. CASE. MIG-29 MODERNIZATION DECISION 



The ultimate goal of military economics is to discover the truth about relations 
between military goals and the economy in the real world. There are many ways to seek 
the truth. Some scholars believe in empirical verification as evidence supported by 
theory; others assert that truth can be discovered only by logical deduction. The methods 
may vary, but the common element is the reliance on models. A situational model for 
methodology evaluation is an independent element of analysis. It can be used for all 
evaluated methodologies without sufficient changes. Moreover such a model should be 
“method neutral,” so it shouldn’t influence the final results by giving advantages to any 
methodology. The best index of effectiveness for a modernization decision is Force 
Potential. This means that a model should have Force Potential as an output. 

A common organizational model that satisfies most present demands is the 
McCaskey model. (Figure 1). First, this model includes all the factors that can influence a 
modernization decision. Second, this model describes the interconnections of the 
elements. Finally, this model allows us to use integrated indexes such as force potential 
for analytical purposes. But, this model is too broad for numerical analysis. Practical 
models are usually derived from methodology and analysis demands. So, what are the 
main features of the Ukrainian Air Force that should be under considered for 
modernization modeling? 
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Air Force Organizational Model 



INPUTS 



TRANSFORMATION PROCESS 



RESULTS 





Figure 1. Air Force Organizational Model 
A. ENVIRONMENT AND DIRECTIONS 

The main elements of the external environment of the Ukrainian Air Force are the 
State, its security formations, politics and the economy. Strategy and doctrine set the 
main directions. 

Ukraine is a non-nuclear state and is defensive in character. At a doctrinal level, 
the Ukraine does not recognize territorial claims upon itself or hold territorial claims on 
other states. The technical aspect of military doctrine emphasizes defense sufficiency. 

Guided by the principles of non-participation in military blocs and alliances, 
which the countiy remains faithful to, the Ukrainian army is now prepared to fight only 
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on its own territory. This really means that the Ukraine, unlike many European NATO 
countries, cannot afford a small or a weak army. 

The Ukrainian military for the last five years have implemented territorial 
principle of defense strategy and new military administrative organization. The next step 
of the military reform will be reorganizing divisional structure to a brigade-battalion 
system. The Air Force will shrink to 10-12 brigades. In addition 600 aircraft and 400 
helicopters will be transferred from combat units to reserve or will be disassembled. 

The state intends to support a technical level of modernization with thorough 
modernization of its existing weapons. The state weapon program announced that the 
lifetime of the main armament will be increased to 10-15 years. Also the procurement of 
new weapons will be minimized. 

Financial support of the army is insufficient. Ukraine’s military budget for the 
year of 2000 is Hr2.4 billion ($480 mil. One dollar approximately equals five hrivnas.) 
Objective data shows that for the last several years the army has been underfinanced. 
Hr 2.4 billion military budget in existing conditions permits weapon R&D plus 
procurement in the level of approximately Hr300 millions. ($60 min.) So, one can say 
that the Ukrainian military has already lost one important campaign - the budget war. 
For the year 2000, they received less than half of what MOD analysts have estimated to 
be their minimum needs. 

The present external environment for Ukraine can be characterized as friendly or 
neutral. On the other hand, all the main neighboring countries have different historical 
claims for different parts of Ukrainian territory. The Ukraine has cultivated a good 
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relationship with NATO. It was the first CIS State to join the Partnership for Peace (PfP). 
There are a number of special agreements between Ukraine, the United States, Russia and 
NATO. These agreements were created to increase state security after Ukraine’s nuclear 
disarmament. 

Ukraine’s economic potential is considerable; it has strong agricultural traditions, 
a large domestic market and trained labor, a diverse industrial base and substantial natural 
resources. However, the economic development has been slow and reforms 
inconsistently applied and frequently subject to bureaucratic interference at both national 
and local levels. The Ukraine depends on imported energy from Russia and 
Turkmenistan. The foreign debt equals $13.5 billion. 

Owing to a new state program, by December 31, 1999, the Ukrainian army will 
have 310,000 serviceman and 90,000 civilian employees. The state armament program 
will include R&D in the areas of a new missile complex for ground forces, a new anti- 
aircraft complex for the air defense and an aircraft modernization program. The division 
of personnel and the composition of Ukrainian armament are given in Figures (2) and (3). 
The Air Force structure is presented in Figure (4). 



Armed Forces (1999) 


Strength 


Total Armed Forces 


326, 000 


Army 


171,300 


Air Force 


124, 400 


Navy 


12, 500 



Figure 2. Division of Personnel in Ukrainian Armed Forces 
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Type 


Armored Vehicles 


Combat Aircraft 


Major Combat Vessels 


Number 


12, 670 


948 
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Figure 3. Armament of Ukrainian Armed Forces 



## 


TYPE 


ROLE 


QUANTITY 


IN SERVICE 


1 


MiG-23 ML,MLD/UB 


Interceptor/Trainer 


100/27 


100/27 


2 


MiG-29-16 


Interceptor 


161 


161 


3 


SU-17 


Ground Attack 


39 


39 


4 


SU-24MP/MR/U 


Bomber 


160/43/48 


160/43/48 


5 


SU-25 


Close Air Support 


34 


34 


6 


SU-27 


Interceptor 


70 


70 


7 


TU-22/m3 


Bomber 


33/29 


33/29 


8 


An- 12/24/26/ IL-76 


Transport 


40/30/30/188 


40/30/30/188 



Figure 4. Main Inventory of Ukrainian Air Force 



B. STRUCTURE AND TECHNOLOGY 

Ukraine’s current adherence to the CFE treaty limits it to 1,090 combat aircraft, 
330 armed helicopters and 100 naval aircraft. Aircraft assets is divided 75:25 between 
the Air Force and the Air Defense. 

According to the new concept, the basic structure of the Air Force will be a 
brigade consisting of 4 to 5 squadrons, and 12 to 14 aircraft each. The brigade will have 
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one basic airfield where all structures are based during peacetime and may have up to 
four reserve airfields for maneuvers. The brigade might have one or two types 
(modifications) of the aircraft of the same class. The normative ratio of pilots to aircraft 
is 1.5:1. Air Force norms say that fighter pilots must have 90 tol 10 flight hours a year to 
maintain or increase their proficiency. Presently the real flight rate of Ukrainian pilots is 
about 40-50 hours a year. This is really a small number in comparison with NATO’s 
standard of 150 to 220 flight hours. 

Each air squadron has its own maintenance command, which is able to work 
autonomously from the base. Squadron engineers and mechanics are able to localize any 
defect and to perform simple repairs and regulation tasks. The brigade technical unit 
provides periodical maintenance (each 100, 200 or 500 of flight hours). The brigade is 
also the main unit for combat-damage repair. Periodical repair (1500 to 3000 flight 
hours), upgrade and modernization are usually handled by air depots - separate military 
units connected to center or territorial command. Every depot has a narrow specialization 
according to the types or models of aircraft. The air industry of Ukraine is able to design 
and produce modem transport and civilian aircraft. There are two major air assembly 
plants, one engine plant and several aggregate, avionics and radio equipment plants in the 
industry. 

C. ALTERNATIVES 

The Ukraine Air Force has 161 MiG-29 fighters. 90 of them (two brigades) are 
aircraft model MiG-29S (design bureau index 9-13) which fit the MiG-29SMT 
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modernization program. All of them have sufficient lifetime and technical conditions to 
be modernized. 

Let us assume that the Ukrainian MOD is in the process of the MiG-29 
modernization program selection. The money available for this program for five years is 
$100 millions. Forty million is earmarked for the first year and $30 million is 
appropriated for the second and third years respectively. 

The Air Force command (1), The Air Force University (2) and the General Staff 
Aviation Planning Group (3) have presented three different alternatives: 

• MIG MAPO MiG-29SMT program (Air Force command). (MAPO is 
Russian abbreviation for Moscowskoe Aviacionnoe Proizvodstvennoe 
Ob’edinenie); 

• MAPO-DASSO program (Air Force University); 

• Intensifying training program without major modernization (General 
Staff). ' 

After the initial discussions are completed, the MOD military board ordered the 
Armament Department to conduct a military-economic analysis of the alternatives. As 
the index of effectiveness, the experts needed to use the integrated combat potential of 
the Air Force brigade for five years. Analysis experts agreed to use two different tools: 
(1) TASCFORM methodology (Timperlake, 1-2) and, (2) Logical Decision for Windows 
(LDW). The Data available for alternative’s assessment are shown in Figure (5). The 
qualitative description of proposed alternatives is given below. 
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N 


PARAMETER 


PHYS. 

DISC 


ALTERNATIVES 


MIG-29s 


Mapo 

Dasso 


Mapo 

SMT 




Service life 


Hours 


4000 


5000 


6000 




Max weapon load 


kg 


4000 


4000 


4500 




Operating weight (empty) 


kg 


10900 


11000 


12500 




Max fuel load 


kg 


4640 


4640 


5600 




Normal T-0 weight 


kg 


15300 


15400 


16000 




Range 


km 


1565 


1565 


2100 




Missile range 


Km. 


50 


50 


110 




Useful air speed 


km/h 


2440 


2440 


2440 




Trust 


kn. 


16600 


16600 


21000 




Maximum take off weight 


kg 


18500 


19700 


21300 




Speed 


Max 


km/h 


2445 


2445 


2445 






S/L 




1500 


1500 


1600 






T/O 




260 


260 


260 




Max rate of climb at S/L 


m/min 


19800 


19800 


21000 




Service ceiling 


m 


18000 


18000 


18500 




Radius of 


800 km/h 


m 


350 


350 


350 




Turn (3.8g) 


408km/h 




225 


225 


225 




Range 


With max internal fuel 


km 


1430 


1430 


2100 






With underbelly tank 




2100 


2100 


2800 






With three fuel tanks 




2900 


2900 


3500 




G limits 


Above M0.85 




7.5 


7.5 


7.5 






Bellow M0.85 




9 


9 


9 




Radar 


Detection range (fighter) 


km 


100 


100 


130 






Tracked targets 




10 


10 


10 






Engage 




2 


2 


4 




Air-to-air 


Long range 








(2) 






Mid range 




2 


2 


4 






Short range 




4 


4 


4 




Air-to- 


Unguided bombs 




+ 




+ 




ground 


Guided bombs 








+ 






Missiles 








+ 




Power plant 


Type and number 




2 x RD33 


2 x RD33 


2 x RD43 






Thrust 


KN 


49.4 


49.4 








Thrust +Afterbum. 


KN 


54.9 


54.9 


98.1 






Lifetime 


Hours 


900 


900 


1500 




Base of avionics 




Specialized 


MIL-STD- 


MIL-STD- 










computer 


1553B 


1553B 




Navigation System 




RSBN 


GPS 


GPS 










RSDN 


Glonass 


Glonass 












RSBN 


RSBN 



Figure 5. Initial Data for Modernization Decision 
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1 . 



The MiG-29SMT Program 



I assume that the fixed cost for both the SMT and the MAPO-DASSO program is 
$10 million. This includes the cost of the technological documentation, any additional 
equipment and personnel training. The variable cost of SMT program is $3 million for 
each aircraft, including the cost of new aircraft equipment, labor and overhead. Lviv’s 
air depot estimates that it would be able to modernize 10 aircraft a year to the SMT level 
after a six-month preparation period. 

The main features of the program are: 

• lifetime prolongation complex; 

• increasing fuel capacity; 

• upgraded radar; 

• a new air-to-air and air-to-ground weapon; 

• new countermeasure equipment; 

• upgraded avionics and cockpit; 

• radar dissipating coating. (Polushkin, p. 6) 

2. The MiG-29 MAPO-DASSO Program 

The variable cost for the MAPO-DASSO modernization is $ 1 million for each 
aircraft. One air depot after a three-month preparation period can provide modernization 
of 30 aircraft a year. The main features of this program are: 

• IFF transponder with all current modes; 
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• A free selectable VHF/UHF communication set, emergency 
communication and collision warning lights; 

• Lifetime enlargement program (1, 100 flight hours); 

• GPS system with a 1553B databus link to an aircraft navigation system, 
plus an enhanced 1553B databus compatible system with an integrated 
laser-internal navigation system. (Mader, p. 3) 

3. Training Program 

Contrary to groups presenting particular modernization programs, the operational 
planners propose to spend any available money to intensify the Air Force training 
program. Their argument is based on the fact that pilots don’t receive sufficient training 
and any investment in aircraft will produce minimum benefits. Combat effectiveness 
depends on both aircraft performance and crew proficiency, so modernization is cost 
effective only with a certain level of pilots training. Presently the average flight time for 
pilot is 40 to 50 hours a year. The data and calculations presented by the General Staff 
group show that the best results can be reached by spending the available money in the 
following ways: 

• aircraft’s lifetime prolongation program for five years - $20 million; 

• spare parts - $15 million; 

• modem training equipment and simulators - $15 million; 

• fuel and materials ( $10 mln./year) x (5 years) = $50 million. 

The cost of fuel appears to be the only variable cost for the flight-intensifying 
program. The Air Force should at any rate spend money for personnel and base 
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maintenance with or without flights. Costs of aircraft and equipment are sunk in this case 
and costs of other materials are significantly smaller than the cost of fuel. 

Aircraft fuel in the Ukraine is about $400 1, 000 kilogram. An additional $10 
million a year remains to buy 2 5, 000 tons of fuel. MiG-29 consumes approximately 3 
tons of fuel in a one-hour flight. This gives the Air Force an additional 8,300 flight 
hours. For each of 130 pilots of two MiG-29 brigades this cost roughly provides 65 
additional flight hours per year for five years. Together with the available flight time this 
number will reach about 100 hours a year per pilot. 



Pilot Proficiency as function of yearly flight rate and flight 
experience 




1 2 3 4 5 

years 



Figure 6. A Pilot’s Proficiency as a Function of Training 

For analytical purposes General Staff analytical group assumes that the 
proficiency of pilots depends on experience and annual training activity. Proficiency here 
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is the percentage of usage the aircraft combat capabilities. Figure (6) shows 
proficiency as functions of flight experience and the yearly flight year rate. 

D. TASCFORM-AIR MODEL 

The Analytic Scientific Corporation (TASC) has developed a method to quantify 
military force modernization based on the measured performance characteristics of a 
specific military system. TASCFORM provides statistic indicators of military force 
potential called measures of effectiveness (MOE’s). The measurements of force 
effectiveness also include quantities of individual weapon systems and are expressed as 
numerical scores. Figure (7) shows the structure of the TASCFORM AIR model results. 

Individual system measures of effectiveness for aircraft are determined by 
comparing performance characteristics such as payload, range, speed, maneuverability 
and targeting to those same characteristics of a specified baseline aircraft. The relative 
importance of these characteristics for each mission is accounted for through by 
weighting factors developed by the panel of experts using Delphi-like techniques. 

Calculated individual weapon systems values can be used alone or they can be 
combined with inventory level and crew proficiency to produce an aggregate theoretical 
force potential. (Regan, p. 1-1) 

Finally it should be noted here that the TASCFORM-AIR methodology does not, 
in its present form, consider cost as input. The model attempts only to measure 
capabilities as an output. Consequently, the measures of effectiveness are not 
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synonymous with cost effectiveness, and care must be taken to avoid reaching 
conclusions relative to cost effectiveness when comparing one system’s measures of 
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effectiveness with that of another. On the other hand, in our problem, an available budget 
is constant for all alternatives. This budget defines a number of aircraft to be modernized 
(MIG-29 MD, SMT), or increased the level of aircrew and groundcrew proficiency due to 
better financing. All these parameters are included in force potential level measures of 
effectiveness. Therefore, in force level we can consider measures of effectiveness as a 
cost effectiveness index. 

A detailed description of the methodology is given in Appendix (A). Excel-based 
calculations of alternatives are given in Appendix (B). 

E. LOGICAL DECISION FOR WINDOWS SOFTWARE 

Logical Decisions for Windows (LDW) software and methodology is based on 
Multiattribute Utility Theory (Appendix C). It helps to evaluate decisions quantitatively. 
Analysts have to define alternatives and their variables. LDW methodology describes 
alternatives by measures, which are numerical or descriptive variables that capture some 
quality of alternatives. 

The measures in LDW are organized under goals; concerns that may affect 
choice. The software helps to organize goals and measures into a hierarchy. LDW uses 
an overall score, called utility, to rank the alternatives. The program computes an 
alternative’s utility by combining its measure levels based on the analyst’s preferences. 
Hierarchy of goals for aircraft modernization is shown in Figure (8). 

The LDW provides considerable flexibility in alternative evaluation; therefore, 
several basic steps are common. First, the analyst must structure the problem as a 
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Figure 8. LDW Modernization Model’s Hierarchy of Goals 
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hierarchy of goals, measures and measure categories. After that the analyst has to 
describe the alternatives. The ability of LDW to transform the data to utilities allows one 
to use a wide scope of physical, financial data and even social values to describe 
competing alternatives. 

The next step is the assessment of preferences. LDW provides a large variety of 
analytical methods to state relations between independent and dependent variables. 

Finally, the program ranks the alternatives and provides the results in a number of 
ways; numerical, graphical and probabilistic. 

The force level utilities in our model are based on multiplicative approach similar 
to TASCFORM methodology and PEREGRINE model (Appendix D). 

Appendix (F) presents short description of different decision support and 
accounting software available for decision-makers. 
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IV. ASSESSMENT OF METHODOLOGIES 



As with any complex subject, the methodology for a modernization decision 
cannot be assessed using only one or two parameters. Three main parts of methodology 
are important for decision-makers. The first part is the initial data required for analysis, 
including constants and variables (input). The second part includes algorithm, 
assumptions and axioms (model). The third part of the analysis is the results developed 
by the methodology to make or support decisions (outcomes). 

In addition to comparing these elements, I will try to analyze the application of 
methodologies to a given problem, the inherent subjective factors of both methods and 
the influence of these factors on the model and on the results. 

A. INITIAL DATA 

The initial data for modernization decision are numerous aircraft characteristics, 
such as performance and mass-geometric data, indexes of effectiveness, economy and 
reliability factors, lifetime resources, etc. 

One can distinguish two situations with aircraft modernization decision-making 
relevant to initial data. The first is when a modernization decision is based on data of 
existing prototype, and second - when the decision is based on the estimated data of the 
modernization program. In the first case the decision-maker has more or less reliable 
statistical data of the aircraft’s physical performance. In the second case, an analyst 
should deal with estimates of data and performance that are usually described with 
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certain probabilities. Since the main task of this thesis is methodology assessment and 
comparison, I will assume that the availability and the accuracy of the data used in the 
thesis computations are equal to the data available for MOD level of analysis. Available 
numerical data are sufficient for both TASCFORM and LDW methodologies. 

There are several sources of data used by MOD analysts for aircraft 
modernization decision. The first source is the design bureau manuals and technical 
reports. Most countries that produce aircraft have state flight-test centers. Their test 
reports usually have priority above all other sources. For modernization program 
selection purposes, in the case a prototype doesn’t exist, analysts can use additional 
independent sources such as scientific research and publications, specialized journals and 
the estimations of experts. Common data describing modernization programs and 
status quo are given in Figure (5). 

B. ADITIONAL DATA REQUIRED BY METHODOLOGIES 

Scope of the data actually used by analysts is much wider than data given in the 
table. For example, to estimate the performance of an aircraft’s radar one should 
consider several dozens of parameters: signal power, sensitivity, weight, accuracy, 
reliability, countermeasure sustainability, weight, cost, maintainability etc. With more 
details, one can assess numerous characteristics for each element. For instance detection 
range of the radar can be described for high, middle and low altitudes, for different 
weather conditions and for various landscapes under the aircraft. In addition, target speed 
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and relative target position can influence detection range. This shows that detailing of the 
model may significantly complicate any problem. 

Therefore one of the most important problems for decision-makers is to agree on 
is “How much initial data is enough?” To make such a decision the analyst has to make 
several assumptions and approximations, which help to simplify the problem. So, what 
are the main assumptions and additional data required by the TASC and LDW models? 

1 . TASCFORM Air Model 

The TASCFORM Air Model generates measures of effectiveness for individual 
aircraft and for tactical Air Forces. Appendix (A). 

There are four types of data required by this model: 

• Performance numerical data; 

• Expert’s weighting factors; 

• Model constants; 

• Data of the aircraft that is used for normalization. 

Performance in this model is actually the median of various data received and 
estimated by design bureaus and by the testing centers. All this data can be described by 
the distribution functions. 

A weapon potential score is computed by scoring the airframe, power plant and 
payload characteristics and normalizing that score against a baseline aircraft, the U.S. 
F-4B. Subjective weighting factors (Figure 9) assign relative importance to the 
characteristics depending on how they contribute to the air combat or surface attack role. 
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In cases where a new model of an existing aircraft possesses a significant change in 
capability (MiG-29md/smt), that model is treated as a separate aircraft. 
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Figure 9. TASCFORM AIR Model Weighting Factors and Constants 
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presents all the data used and the results of the TASCFORM- AIR model for MiG-29 
modernization decision. 

2. LDW Aircraft Modernization Model 

In contrast to the TASCFORM AIR Model, the LDW method doesn’t propose 
specialized algorithm for any specific problem. It proposes a method and universal 
software for most decision-making problems. A detailed description of the LDW isn’t 
the subject of this thesis, so I’ll emphasize only the main points of the solution and 
qualities of the methodology relevant to my comparison with the TASCFORM AIR 
Model. 

Actually LDW allows one to replicate the TASCFORM approach and to obtain 
the same results. It can use all the formulas of the TASCFORM AIR Model and its 
weighting factors. But it is not the most convenient way to use LDW. 

In the modeling modernization decisions one can conclude that there are only 
certain specific factors, which are really influenced by modernization. So, an effective 
solution may not include all the factors and characteristics of the aircraft but rather, just 
the ones that actually change. For comparison purposes, Adjusted Aircraft Potential 
(AAP) is herein selected as the main goal for LDW modernization model. (Figure 8) 
Similar to the TASCFORM Adjusted Weapon System Potential (AWSP), AAP includes 
several lower level goals such as aircraft potential and availability. In contrast to 
TASCFORM, AAP also includes economy factors and remaining age index. 
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LDW converts the alternative’s measure levels into common units - utility. The 
highest utility is equal to one, the lowest is equal to zero. Intermediate levels are 
assigned with what is called a Utility Function, or more specifically a Single Utility 
Function (SUF). The easiest representation of SUF is a straight line. This is the most 
common case, and the default. If straight lines SUFs aren’t appropriate, any kind of 
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Figure 10. Data Assigned for LDW Aircraft Modernization Model 



48 



Utility 



0 




MAINTAINABILITY index (index) 

Preference Set = best fighter 1 



REMAINING AGE index (index) 
Preference Set= best fighter 1 



Figure 11. Forms of Single Utility Functions Used in LDW Modernization Model 

monotone non-linear function can be applied. Figure (1 1). 

On the next step LDW computes a goal utility by combining the utilities for each 
of its member in a weight average. Similar to the TASC weighting factors the LDW 
model member’s weight determines how much it will influence an alternative’s utility for 
the goal. LDW provides several methods for assessing weights. The simplest method 
uses a simple ordering of the measures’ relative importance. Other methods, such as 
“smart” and “analytical hierarchy process” have analysts enter the ratios of the 
measure’s relative weights. Another method called “tradeoff’ is based on the idea that 
equally preferred alternatives should have equal overall utilities. 

Finally LDW provides many ways to review results. It can rank the alternatives 
overall goal or a measure. LDW sorts alternatives by utilities and displays them along 
with a bar representing a relative score. 
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c. 



APPLICATION OF METHODOLOGIES 



The pursuit of efficiency in the military requires a priority to be given to those 
programs, that provide the greatest benefit per unit of cost. Alternative actions often have 
to be compared to determine whether a change in the mix of actions would increase 
efficiency. Also the economic evaluation approaches the cost in a common format, yet 
they differ in the ways they approach benefits. These differences play a critical role in 
developing criteria for efficiency. 

So, what aspects of the models do influence the methodology selection, and what 
aspects of the methodologies influence the selection of criteria? 

1. Hypothesis 

Two main hypotheses have influenced the selection of methodologies and criteria 
for a modernization decision. The first is that effectiveness of the aircraft is primarily a 
function of the aircraft potential and the pilot proficiency. Therefore, investing in the 
aircraft’s modernization is cost-effective only from a certain point of crew proficiency. 
The second hypothesis is that the varying money available for modernization may change 
the nature of the best solution. In this case it depends on the difference between marginal 
benefits of modernization and training. 

These hypotheses firstly led to including the status quo plus training alternative 
for consideration. Secondly, the assumed hypothesis predetermined the choice of the 
force level indexes of effectiveness as criteria. Finally, both hypotheses provide a better 
understanding of the real goals of economic analysis in the military. 
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2 . 



Criteria of Efficiency 



Efficiency in military resource allocation matters whether these resources are 
being used to get the best value for money or not. Efficiency is concerned with the 
relation between resource inputs and either intermediate outputs ( in our case aircraft 
potential) or final military outcomes (force potential,' readiness, etc.). Also many 
evaluations use intermediate outputs as a measure of effectiveness, this leads to possibly 
suboptimal recommendations. Ideally, economic evaluation should focus on final military 
outcomes. 

There are three main concepts of efficiency in the theory: technical, productive 
and allocative. Technical efficiency addresses the issue of using given resources to the 
maximum advantage; productive efficiency of choosing different combination of 
resources to achieve the maximum benefits for a given cost; and allocative efficiency of 
achieving the right mixture of military programs to maximize force potential. Although 
productive efficiency implies technical efficiency and allocative efficiency implies 
productive efficiency, none of the converse implications necessarily hold. Faced with 
limited resources, the concept of productive efficiency will eliminate as “inefficient” 
some technically efficient resource input combinations, and the concept of allocative 
efficiency will eliminate some productively efficient resource allocations. (Palmer, p. 2) 

TASCFORM-AIR Model generates measures of effectiveness for the individual 
aircraft and based on it - for the tactical air forces. As far as these indexes are ratio 
comparison of evaluated and basic aircraft, we can believe the results of TASCFORM 
evaluation is appropriate for cost effectiveness analysis. This cost effectiveness analysis 
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type of analysis measures military benefits in constructed indexes (weapon potential, 
force potential). Since costs and benefits are measured in non-comparable units, their 
ratios provide a yardstick with which to assess relative (productive) efficiency. This 
decision rule does not, however, enable us to evaluate the relative efficiency of actions, 
which provide more benefits at a greater cost or less benefits at a lower cost. A major 
limitation of cost-effectiveness analysis is its inability to compare alternatives with 
different natural effects. For example investments in air defense cannot be directly 
compared with those that improve air forces. Cost-effectiveness analysis therefore 
cannot directly address allocative efficiency. 

Logical Decisions for Windows provides us with utilities for individual aircraft. In 
the case of linear approximations for single-utility functions and with the absence of 
probabilities in the model the result has an interval scale meaning. Force level measures 
of effectiveness are the results of the multiplication of an aircraft potential by the pilot’s 
proficiency and inventory level. It lets us consider the aircraft potential and force 
potential based on LDW calculations as an index for cost-utility analysis. Generally 
cost-utility analysis is an adaptation of cost effectiveness analysis which measures the 
alternatives effect on both the quantitative and qualitative aspects of military indexes 
using a utility based measure. Like cost effectiveness analysis, relative efficiency is 
assessed using an incremental ratio, here a cost-utility ratio. An alternative is deemed 
productively efficient, relative to another one, if it results in higher (or equal) benefits at a 
lower cost. The use of a single measure of military benefit enables diverse alternatives to 
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be compared so cost utility analysis can address both productive efficiency and allocative 
efficiency. 

In cost-utility analysis the optimal decision rule involves ranking the incremental 
cost-utility ratios of different alternatives and selecting those with the lowest ratios (best 
value) until the budget is depleted. The lower the incremental ratio, the higher the 
priority in terms of maximizing military benefits derived from a given level of 
expenditure. The point at which resources are exhausted defines a maximum price for a 
unit of effectiveness. Eliminating interventions with an incremental cost above this price 
in favor of those with lower incremental costs would be considered an improvement in 
allocative efficiency. (Palmer, p. 3) 

3. Adjusting the Methodologies to the Model 

Our situational model can be described as a model of the Air Force unit which 
consists of two brigades armed with MiG-29 (90 aircraft). Environment, directions, 
constraints and culture are common for all alternatives. The difference is in the “process” 
and “structure.” Brigades may increase their combat potential by modernizing aircraft or 
by intensifying training. 

The TASCFORM-AIR model doesn’t demand any structural changes or 
adjustments to presented model. To construct the Equivalent Force Potential (EFP) 
measure of effectiveness, the following assumptions should be made: 

• Structure of Brigade for each year shall be calculated in accordance with 
the scenario. For example, for the status quo alternative the number of the 
aircraft constantly equals 90 MiG-29S. For SMT and MD alternatives it is 
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a mix of the modernized and basic aircraft, which depends on the air depot 
productivity. 

• Concept of pilot’s proficiency in the model is based on assumption that a 
fully-trained pilot can use 100% of the aircraft potential. To achieve this 
result a pilot needs five years of intensive training (120-130 flight hours a 
year). A yearly flight rate of 50 hours let pilots keep theirs proficiency 
on the 50% level. For the five-year period, I assumed the linear 
approximation of proficiency function. 

Pl(t) =0.5 Flight rate 50 hours/year; 

P2(t) = 0.09* Y + 0.5 Flight rate 120 hours/year. 

Y = {1, 2, 3, 4, 5}, year of program. 

Use of more complicated proficiency functions shouldn’t create significant 
difficulties. 

As mentioned earlier, the LDW is a methodological framework for decision 
support. Using LDW, the analysts create their own structure and logic for the problem 
based on their model and intentions. The basic structure of goal hierarchy for a 
modernization decision is based on the Grippen concept of aircraft effectiveness 
(Appendix F), adjusted TASCFORM weighting factors and PEREGRINE multiplicative 
approach for the force potential level of effectiveness. (Appendix D) 

In our case, modernization influences a comparatively small number of aircraft 
performance measures. It helped to create a hierarchy with a relatively small number of 
measures and measure categories. (Figure 12). As far as the main purpose of this thesis 
is the evaluation of methodologies, not the evaluation of modernization alternatives, I 
assigned reasonable utilities and measures in the model where it was not available from 
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the references. The concept of force structure and pilot’s proficiency remains the same as 
in the TASCFORM-Air model. 
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Figure 12. 



LDW Modernization Model Goals Hierarchy. 



Overall Weights. 
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D. THE RESULTS 



Methodology selection demands a special approach to the comparison of the 
results. The main questions usually are, “What are the forms of the result? What does it 
mean? How is it sensitive to changes in inputs? Do both methods support similar 
decisions and prove initial hypothesis?” 

1. The Form and the Meaning of the Results 



The TASCFORM-Air model generates measures of effectiveness for individual 
aircraft. Figure (12) shows the table of results for individual levels: Weapon System 
Potential (WSP), Adjusted WSP and Designated AWSP. 
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Figure 13. TASCFORM-Air Model Individual Measures of Effectiveness 

The TASCFORM-Air model compares a given aircraft with the basic aircraft (F- 
4B). It does not directly address the military utility of the basic or evaluated aircraft. 
The meaning of the TASCFORM scores is actually the rate that one is willing to 
substitute one aircraft to another in order to have the same mission efficiency. The score 
of the basic aircraft in the model is equal to 10. The results in Figure (13) show us that 
individual scores for MiG-29S/ MiG-29MD/ MiG-29SMT programs are rated as 
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18.1/ 20.3/ 28.5. This means, for example, that one M1G-29MD is equal to two F-4Bs in 
the interceptor role and one MiG-29SMT is almost equal to three F-4Bs. 

Computing the measures of effectiveness of air forces (or any air force unit) is a 
two step process. First, a designated force potential score is computed. This score is the 
sum of scores of the individual aircraft performing a given role. (Timperelake, pp.. 2-3) 
Next, an equivalent force potential measure of effectiveness is computed by scoring 
relative obsolescence, multi-role capability, the effect of the numbers, C3I capabilities, 
logistic and personnel qualities. 

Air force level indexes of effectiveness (Designated Force Potential and 
Equivalent Force Potential) have meaning only for a particular situation. In accordance 
with the model and the problem design it may be valuable for resource allocation tasks or 
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Figure 14. TASCFORM-AIR Model Force Level Measures of Effectiveness 
rational force structure definition. Graphs of the force potential measures for MiG-29 
modernization problem are given in Figure (14). 
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As we can see, TASCFORM AWSP and DFP scores for the MiG-29 
modernization alternatives give preference to the SMT program, while the EFP shows 
that the training program by the end of a five-year period is 20 % more effective than the 
“SMT” program. In our case it means that for the same money ($ 100 million), training 
gives 20% better result than modernization. 

In contrast to the TASCFORM- AIR model, the LDW model provides only one set 
of results. It is the utility for assessed alternatives. LDW computes an alternative’s utility 
by combining its measures level based on the analyst’s preferences. The most common 
forms of the results are given in the Figure (15). 



Rank in g for ADJUSTED A IR CRAFT POTENT IAL Goal 



Alternative Utility 



MIG-29SMT 0.712 

MIG-29MD 0.506 

MIG-29S 0.370 



Preference Set= best fighter 1 



Rank in g for ADJUSTED A IR CRAFT POTENTIAL Goal 



Alternative Utility 



M IG -2 9 S M T 
M IG -29M D 
M IG -2 9 S 



0 .7 1 2 
0.5 06 
0.370 




| ~ " l&awaSi# 



H AIRCR. POTENTIAL^ AVAILABILITY ^ ECONOMY FACTOR 

S REMAINING AGE index 



Preference Set= best fighter 1 



Figure 15. LDW Ranking of Alternatives 
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ADJUSTED AIRCRAFT POTENTIAL 




Figure 16. Graphs of the LDW Model Adjusted Aircraft Potential 

Time can be incorporated in the LDW model, with separate program runs done 
for each year. Figure (16) shows the graphs for alternative utilities in a time perspective. 

The advantage of LDW is a wide range of choices of methods to provide and to 
analyze the available results. There are sixteen different functions in the LDW result 
mode. It can be pure “rank alternative” or “stacked bar ranking” that show relative 
importance of each goal in the final result (Figure 14), or it can be diagrams or tables 
explaining the different aspects of the result. 

In contrast to TASCFORM, LDW computes an interval scale. Changes in inputs 
give changes in utilities, and those changes can be compared. For individual aircraft we 
can compare changes in utilities from basic level (status quo) to the variable cost of 
modernization. Cost-utility ratio (CUi) for “SMT” and “MD” alternatives can be 
computed as : 

CUi = ( Ui - Uo )/ Ci, where; 

Ui, Uo - Utilities of the assessed alternative and status quo; 

Ci - Variable cost of the modernization program. 
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The force level indexes in the LDW model is based on the same data used for the 



inventory level and the crew proficiency, as was the case in the TASCFORM-AIR model. 
The LDW force level utilities are given in Figure (17). 




Figure 17. LDW Utilities for the Force Level 

The LDW results are similar to the TASCFORM results at both the individual and 
at the force levels. Both models show that the training program is more effective at the 
force level, while at the individual level the “SMT” modernization program is 
significantly better than all the others. Actually these results prove our first hypothesis: 
If the level of crew proficiency is low - the investment in aircraft modernization (without 
intensifying training) is not the best. 

The assessment of the budget influence led to a predictable conclusion: Every 
program has limited abilities to increase efficiency by intensifying training. From a given 
level of investment, additional training cannot produce additional efficiency. If the air 
forces have reached normative training level, the only way to increase the force potential 
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Influence of the Budget on the Programs' Efficiency 




Figure 18. The Influence of Budget on Program Efficiency 



is to increase the weapon potential. Figure (18) shows an equivalent force potential as 
functions of the budget. 

This graph shows that the training programs actually have reached the maximum 
efficiency on the level of the initial budget. The assumption in the calculations of the 
budget influence is that at first money goes to modernization rather than to training. 
The next results show that doubling the budget increases the efficiency of the “MD” 
program by two times. But employing the “SMT” program with four times the initial 
will result in the highest efficiency rate possible in the situation. 
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In the modem high-competitive military environment, the price to improve 
efficiency is significant. Moreover, the initial choice of program will define the 
maximum achievable level. 



2. Correlation of the Results 



The correlation of the results for both programs is obvious. First, both methods 
have similar results at the individual weapon level. Second, they have the same 
tendencies in transition from one level to another. Finally, changes of similar parameters 
cause comparable changes in the results. 
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Figure 19. Comparison of the LDW and the TASCFORM Individual Indexes 
Figure (19) shows the comparison of the LDW Adjusted Aircraft Potential (AAP) 
with the TASCFORM-AIR model Adjusted Weapon System Potential (AWSP). 

The alternatives in both methods are ranked in the same order with similar ratio 
tendencies. The mutual ratios of the indexes are 1.92/ 1.36/ 1.00 for LDW and 1.57/ 
1.13/ 1.00 for TASCFORM-AIR model. Differences in the ratio are most probably 
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caused by the initial approach to the problem. The LDW considers only the parameters 
that are influenced by modernization, while TASCFORM includes all the aircraft’s 
performances. This makes the difference in the TASCFORM results less then the 
differences in the LDW results. 

Both methods define training (MiG-29S alternative) as the best alternative in the 
force level of analysis. The second and third preferences are different. For example the 
TASCFORM approach evaluates “SMT” program 10% higher than the ”MD” program, 
while the LDW gives 3% preference to the “MD” over the “SMT” alternative. 

Attempts to analyze cost-utility and cost-effectiveness ratios for “SMT” and 
“MD” alternatives at the individual level led to controversial results. Figure (20) shows 
the results of the calculations. 



Alternative 


Cost-Utility Ratio 


Cost-Effectiveness Ratio 




Uo (MiG-29S) 


Eo (MiG-29S) 


Eo (F-4B) 


SMT 


0.114 


3.47 


6.16 


MD 


0.136 


2.2 


10.3 



Figure 20. Cost-Utility and Cost-Effectiveness Ratios of Modernization Programs 

When utilities of status quo (MiG-29S) were used as the basic for calculating the 
cost-utility ratio, the “MD” program had a higher ratio than the “SMT” program and is 
therefore more preferred. The same approach in the cost-effectiveness analysis gives the 
opposite result. Here the “SMT’ program had a better ratio. With basic aircraft changes 
to the F-4B, this ratio once again favors the “MD” program. These results are good 
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illustrations of complexities in the decision-making process. Different indexes and even 
different concepts of efficiency may lead to different conclusions. In our models the 
“SMT” program has the best indexes at the individual level of analysis. The “MD” 
program has a better cost-utility and cost-effectiveness ratio. And finally, the training 
program has a better perspective in the force level. 

3. Sensitivity and Reliability 

Sensitivity analysis is an attempt to deal with uncertainty. Sensitivity analysis can 
be preformed with respect to both predictions of impacts and their valuation per unit of 
impact. In particular, it should convey how sensitive predicted indexes or utilities are to 
changes in assumptions. If the ordering of alternatives does not change when we 
consider the range of reasonable assumptions, then our analysis is robust and we can have 
greater confidence in its results. 

The results of a partial sensitivity analysis for TASCFORM-AIR model are 
given in Figure (21). Partial sensitivity is most appropriately applied to what the analyst 
believes to be the most important and uncertain assumptions. For porpoises of 
methodology comparison, I’ve conducted sensitivity analysis for both groups of the 
results in every method. 

Compared to TASCFORM, the LDW has great built-in possibilities for 
sensitivity analysis. The “Sensitivity Graph” option let us see the effect of changing the 
weights of a single measure or goal. Sensitivity tables let us see the effects of the 
changes in weights of the goals and measures quickly. (Figure 22). 
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# 


PERFORMANCES 


CHANGE 


INFLUENCE ON FINAL INDEXES 






(%) 


AWSP 


EFP 


1 


RANGE 


+20 


+2.5% 


+2.0% 


2 


PAYLOAD 


+20 


+9.8% 


+2.7% 


3 


SPEED 


+20 


+8.5% 


+2.1% 


4 


MANEUVERABILITY 


+20 


+21.5% 


+1.8% 



# 


GOALS/ MEASURES 
WEIGHT. FACTORS 


CHANGE 

(%) 


INFLUENCE ON FINAL UTILITIES 


PERFORM. 


WEIGHT. 

FACTOR 


1 


Range/ WF 


+20 


+0.5% 


+0.15% 


2 


Reliability/ WF 


+20 


+0.25% 


- 6.3% 


3 


C3I/ WF 


+20 


+1.0% 


+1.6% 


4 


Remaining age/ WF 


+20 


+5.9% 


+2.2% 



Figure 21. The Results of the TASCFORM and the LDW Sensitivity Analysis 
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MIG-29SMT 

MIG-29MD 

MIG-29S 



Preference Set= best fighter 1 




MIG-29SMT 

MIG-29MD 

MIG-29S 



Preference Set = best fighter 1 



Figure 22. LDW Sensitivity Tables 
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Reliability of the results in both methods depends on the quality of the initial data, 
the model validity and the quality of expertise. Nevertheless, both methods are useful for 
certain conditions and within the constrained data fields. For example, let us assume for 
both models that the meaning of the range and survivability of the “SMT” alternative is 
equal zero. In reality such an aircraft has no military value. But both of our models still 
assign positive indexes to it. (Figure 23) In practice this means that analysts using these 
methods should understand the nature of the problem and perform a pre-modeling data 
analysis. They need to exclude alternatives with invalid characteristics or modify the 
model to make it reliable. The LDW software has the possibility to include probabilities 
and uncertainties into the analysis. In this case we can estimate the reliability of the 
results numerically, but the results shift from interval to ordinal scales. This makes them 
impossible to use for valuable cost-utility ratios. 

While the nature of the problem and the data available have certain levels of 
accuracy and reliability, every cost-effectiveness and cost-utility analysis should be 
subjected to tests of their sensitivity to the assumptions that they employ. 
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Figure 23. Indexes and Utilities of Alternatives When Speed and Range for 
“SMT” Alternative are Equal Zero 
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D. METHODOLOGIES 



1. Basic Perspectives 

Both the TASCFORM AIR Model and the LDW Model provide the 
dimensionless measures of the relative technological quality of a given system. Both 
employ utility functions, using subjective inputs to assign weighting factors to 
measurable attributes. In formulating the LDW Model, I assumed that analysts might use 
different methods to select the initial data. It may be Delphi approach that depends on 
the written survey data gleaned from experienced military users or from a simple expert 
estimations. TASC, on the other hand, created the TASCFORM-AIR model internally 
and had the preliminary version reviewed by tacair community representatives for 
comments on all subjective aspects. (Timerlake, pp.. 1-5) 

The different methodological approaches and applications of the two models 
dictate different levels of details. By design, TASCFORM is a relatively simple model. 
This limits the volume and classifications of the data to be used. The LDW model based 
on the scenario at hand and on the level of detail included may use significantly smaller or 
greater data. 



2. Outputs and Applications 

Understanding of the TASCFORM-AIR model results for an individual weapon is 
simple. The score for the basic aircraft is 10, so if the assessed aircraft has scored 20, it 
means that this aircraft is equal to two basic aircraft for a certain mission. To understand 
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force level measures of effectiveness, the user should know the scenario. Due to the 
initial data, results may vary from hundreds to thousands. Particular results have 
meaning only for a comparison of the given alternatives in certain situation. 

The LDW results are utilities scored in the interval from 0.0 to 1.0. In accordance 
with the scenario and methods used to quantify preferences, it has (in our case) an 
interval scale meaning. 

The ability to generate scores to measure the potential of individual weapon 
systems (WSP, AWSP) and basic military organizations (EFP), and to modify these 
scores to take into account qualitative factors make TASCFORM useful in a number of 
ways. First, it is useful for doing systems analysis studies to track technology growth 
and cost performance trends. Second, it is useful to do macro-level force balance 
comparison for strategic and operational planning. Third it is useful for conducting net 
assessments of current and future military balances of any region in the world. 
(Chemiavsky, p. 4) TASCFORM and LDW results are intended primarily for the: 

• Static comparison of the aircraft potential; 

• Force potential input into probabilistic dynamic wargaming models; 

• Input into force development and procurement decisions; 

• Measurements of relative force modernization, principally at the total 
force level, defined as the differential of force equipment potential over 
time; 

• Estimation of associated military investments, when TASCFORM outputs 
are coupled with appropriate cost data. 

TASCFORM- AIR Model is able to generate tacair modernization rates. (Regan, 
p. 4-1) These are based on either Designated Force Potential (DFP) or Equivalent Force 
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Potential (EFP), depending upon whether the analyst chooses to assess the extent of the 
modernization embodied in a tacair force when incorporating synergetic factor. 

Modernization rates for the period are calculated as follows: 

AMRc= dDFP/dt = [(DFPi - DFPo)/DFPo] *100% 

FMR = dEFP/dt = [(EFPi - EFPo)/EFPo] *100% 

Where: AMRc- collective aircraft modernization rate; 

FMR - force modernization rate. 

The model thus provides the analyst with the means for assessing the effect of 
equipment modernization. 

3. Form of the Utility Function 

The TASCFORM-AIR Model has four basic additive parameters: payload, range, 
maneuverability, and useful speed to describe aircraft performance. The values used for 
this are actual performance figures normalized to baseline aircraft values. On-board 
systems and features modify these values multiplicatively, and survivability enters as a 
multiplicative modifier of the entire weapon system measures of effectiveness (MOE) 
rather than as an additive term. Individual aircraft MOEs are adjusted for relative 
obsolescence and productivity prior to being multiplied by inventory levels and summed 
to determine the basic force MOE. This is then adjusted by factors reflecting the impact 
of changing force levels, C3I, and multi-role capability. 

The presented LDW Aircraft Modernization Model is based on three basic 
additive goals: aircraft potential, crew potential, and economy factor. Each of these is an 



71 



additive function, which incorporate as many as ten measures and ten more measure 
categories. All measures and measure categories are subjectively measured. The LDW 
utilities for the Adjusted Aircraft Potential are determined by converting given measures 
to common units (utilities) and establishing the relative importance of the goals and 
measures. To receive an index (FPI) comparable with the TASCFORM Equivalent 
Force Potential, one can multiply the AAP by the inventory level. 

4. Scenario Applicability 

The TASCFORM and the LDW approaches are consistent in the area of scenario 
applicability. While the generic TASCFORM methodology was deliberately developed 
with no specific scenario in mind, it had always been recognized that tailoring the 
weighting factors to fit a given scenario is an appropriate application of TASCFORM. 
This is regarded as one of the virtues of the model. (Chemiavsky, p. 5) Similarly, 
different LDW models can be generated using specific scenarios, simply by adjusting 
goal hierarchy and preferences as appropriate. 

5. Treatment of Intangibles and Design Obsolescence 

The basic LDW methodology may include consideration of human factors and 
intangibles in its models. Obsolescence may be expressed as usual utility with negative 
utility function. The given model results are time-specific only in the sense that an 
updated set of scores reflects the consensus of the study group at the time of the update, 
and changes may be manifested in a revised subjective weighting schedule. We can 



72 



conclude that score for the same aircraft generated in different modifications of models 
should not be compared for analytical purposes. 

TASCFORM also contains, albeit in rudimentary form, consideration of some 
intangible aspects of force modernization, such as supportability and maintainability of 
systems, the effects of embodying greater quality in fever systems, and vice versa. In 
addition, it contains the effect of building multipurpose versus specialized systems, the 
effect of aircrew and groundcrew proficiency, the effect of improved force command, 
communication and control, and design obsolescence. The fact that all of these aspects 
change over time is accounted for, and that will enable the TASCFORM user to compare 
the effects on individual systems over time, as well as to assess force-level trends in 
modernization. 

6 . Enhanced Assessment Capabilities 

The given examples of methodology use are limited to weapon system potential 
and theoretical aggregate force potential. In many cases of military related planning and 
analysis these indexes are not sufficient. Determining real force potential requires 
assessing the intangible non-weapon aspects of a military force. The Analytic Scientific 
Corporation has developed a set of companion models to measure the following (Regan, 

pp.. 1-2): 

• C3I systems; 

• The potential of available manpower to operate and maintain a force’s 
weapon systems; 
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• Effects of logistic organization, support, and sustainability on overall force 
potential; 

• Ability to mobilize and deploy a force’s assets; 

• Force structure and resource (fiscal) allocations; 

• Modernization through technology development. 

Compared to TASCFORM methodology, the LDW is a more universal tool. It 
helps analysts evaluate any decision quantitatively. One should define all alternatives 
and variables to describe them. After that, LDW helps to formulate preferences about the 
variables and uses the information to rank alternatives. So the main difference is that 
TASCFORM is a set of exact algorithms to be used in specific areas, while the LDW is 
universal methodology to build different algorithms. In both methodologies, the quality 
of results depend on the quality of initial data and assumptions. TASCFORM models 
are likely to be used widely by the public, because all the expert’s estimations are 
included, while the LDW models are strongly dependent on the quality of “in-house” 
expertise. 

F. LIMITATIONS 

All analytical models have limitations. TASCFORM and LDW are static 
assessment models and have many of the limitations of static models. Both models are 
not predictors of combat outcomes. They are indicators of force potential. In given 
forms and programs’ design both models do not account for dynamic interactions and 
cannot measure the synergy between and among systems in combat. 
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It was recognized early that the availability of data concerning aircraft 
performance and qualities may vary significantly for different modernization programs. 
The limit of methodological details in both models let to the development of relative 
indexes of aircraft and force potential, not an attempt to create unnecessarily precise 
absolute values. 

G. SUBJECTIVE FACTORS 

Several areas within modernization decision process are influenced by 
organizational and individual subjective factors. The most obvious factors are the 
methodology selection, the collection of the initial data, the assigning of weighting 
factors, and finally the subjective political preferences. Therefore, different methods 
may give different results, so the methodology selection plays a certain role in the 
decision making process. In the real world, it is often a matter of competition between 
scientific or research institutions for resources and programs. In our case the TASC 
Corporation has more authority, experience and contacts within the MOD. This 
organization promotes and advertises its products at all levels of MOD organizations. It 
may indeed influence MOD decisions more than any new scientific company just in the 
market. On the other hand, the more methods are used for the assessment, the less is the 
probability of mistakes. 

The selection of the initial data also may influence the final decision. It is clear 
that organizations representing modernization programs will collect and present selected 
data to show their product in a good light. In this case, the strict and stable TASCFORM 
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Air Model is less influenced by the initial data than the LDW models constructed for a 
particular problem. But data collected from independent sources (if available) may 
eliminate this problem. Weighting factors are the most important and most subjective 
part of both the TASC and the LDW methodologies. The TASCFORM Air Model 
includes 12 individual weighting factors, while the presented LDW model consist of 17 
expertly assigned utilities, 10 category multipliers, and 17 SUFs. All these numbers 
represent the expert’s estimations of data, the relations or the relative importance of any 
presenting factors. 

TASC emphasizes that their weighting factors are the result of a series of 
conferences, seminars and meetings where a wide spectrum of military scientists, pilots, 
engineers and operational planners had made their estimations. These estimations were 
later statistically transformed to weighting factors present in the model. Reliability of 
any model based on expert judgement is a function of the number and qualities of experts 
participating in creating the model. One more significant factor, which influence military 
decisions, is the political preferences. This becomes very important in case of selections 
between domestic and foreign armament. Below are the results of a short questionnaire 
among the Naval Postgraduate School international and US students concerning their 
personal preferences in armament acquisitions. Two questions were posed: 

• Assume that you are the member of a committee for acquisition source 
selection. In particular situation you have three choices: (Select one) 

• To buy domestic weapon; 

• To buy 30% more effective foreign weapon at the same price; 
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• To postpone acquisition for five years and to invest the money in 
R&D with a 60% probability of creating an equaled weapon in the 

future. 

When would you prefer armament produced in a foreign country? (Give 
both figures) 

• Armament produced by allied country is more effective than 

domestic. 

• Armament produced by a non-allied country is % more 

effective than domestic. 




ABC 



Figure 24. Distribution of Responces for Qestion 1 

(A-Buy domestic; B- Buy 30% more effective foreign; C- Invest in R&D.) 

The results of this questionnaire are given in Figures (24) (25). 

From the diagrams we can conclude that weapons that are produced by allied 
foreign countries should be 20% more effective in order to be equally preferred to the 
domestic ones, and 50% more effective if produced by non-allied countries. Reliability 
of these results is not very high because of the small number of participants, but the 
common tendency is obvious: political preferences have a strong influence on any 
military decisions. 
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Figure 25. Distribution of Responses for Question 2 

(Level of effectiveness for foreign weapon to be preferred to domestic.) 



78 



V. CONCLUSIONS 



Making a decision is like playing Mozart on the piano: it is easy to do, but 
hard to do well. (Brenesal, p. 1) 

In today’s corporate world and bureaucratic environment, making a good decision 
in a timely fashion is essential. However complex alternatives, uncertainties, and varying 
goals and preferences render this difficult - and a decision once made, can be nearly 
impossible to explain to people not involved in the decision-making process 
Methodologies, which are assessed in this thesis, allow decisions based on quantitative 
analysis. They also help to understand and to explain the decisions being made. These 
methodologies are intended to bring clarity to the evaluation process. 

The TASCFORM and the LDW methodologies have much in common. Both 
employ utility functions, rely on subjective inputs, and can address different scenarios. 
The differences stem from a fundamental divergence in the approaches to aggregating 
various system attributes. In the TASCFORM approach, the physical performance 
parameters such as range and payload are considered to be the basic, additive parameters 
that establish the system's weapon potential. These basic characteristics are then modified 
by on-board system capabilities, which enlarge or diminish the significance of those 
physical performance parameters multiplicatively. In the LDW approach, on-board 
system capabilities are part of the goal called “combat effectiveness.” Their utilities are 
added to a common function in accordance with the individual SUFs and preferences. In 
the TASCFORM, survivability features are represented by another multiplicative term. 
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which adjusts the overall "productivity" of the weapon system. In the LDW approach, it 
is a separate measure expressed only by countermeasure utilities just because the rest of 
the factors, which influence survivability, didn’t change in the modernization of the 
MiG-29. 

There are possible differences caused by the strictly non-linear translations of 
performance capabilities into additive component indices in the LDW approach. 

This does not imply, of course, that there is no correlation between LDW and 
TASCFORM. The rank ordering of the alternatives, for instance, is the same with the 
LDW or the TASCFORM. Both approaches consider virtually the same set of weapon 
system attributes and grant them generally the same order of importance. The difference 
is that the range in the indexes between the most and least capable aircraft is 1.9:1 by the 
LDW, but only 1.57:1 by the TASCFORM 

The TASCFORM-AIR Methodology does not consider cost as input, so the 
measures of effectiveness are not synonymous with cost-effectiveness. However, it 
should be noted that force level measures of effectiveness for a given air unit are 
subsequently multiplied by the number of that aircraft’s inventory at a selected time to 
derive its contribution to an overall force measure of effectiveness. Those inventory 
levels are related to system costs, thus introducing a second order cost effect in the 
model. 

The TASCFORM-AIR model requires knowing a large number of physical 
parameters. The weighting factors and constants for the main tactic aircraft missions are 
included in the methodology. It is a well-developed model with a comparatively specific 
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application range. It makes this method easy to use by any specialist in any level of 
military planning. Because the model is framed, the analyst has to adjust the situation to 
the model. 

The LDW methodology requires a good understanding of the problem at hand and 
a high level of expertise in order to create structure (the goal’s hierarchy) and to define 
preferences. The absence of the strict framework gives wide possibilities, but demands 
experience and accuracy. LDW is a highly adaptive tool based on multi-attribute utility 
analysis that is good enough to be used in most decision support systems. This method, 
in contrast to the TASCFORM, may be adjusted to any situational model. 

The results produced are useful in several ways, but as indicators rather than as 
"answers." First, the quality/quantity index values for weapon systems and forces can be 
used to analyze the effects of technology growth and modernization measures on the 
conventional force potential. These can then be compared to similar indicators to assess 
ranges of uncertainty regarding these trends. Second, the index values are quickly 
generated and the system can therefore be rapidly reprogrammed to generate alternative 
models based on different performance assumptions. This ability to offer comparisons of 
the effect of different perceptions is useful in suggesting directions for more detailed 
analyses using more complex simulation models and techniques. 

It is important for an analyst to reflect the effects of product improvements, given 
the extent to which modifications such as electronic countermeasures, improved 
munitions and new navigation systems have been added to existing weapon systems. 
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TASCFORM caters to this and can depreciate quality indexes with time in the absence of 
improvements to reflect the relative decline in potential of older weapon systems. 

The form of the TASCFORM results is more convenient and understandable for 
the user. It has the ratio meaning and can be easily used to compose different indexes. 
The understanding of the LDW resulting utilities is more complex. In accordance with 
the applied forms for the single utility functions and applied methodology of preferences, 
utilities may have either an interval or an ordinal meaning. This fact makes the LDW 
utilities difficult to use in the cost-utility ratios. 

We have to mention here that both methods support the initial hypothesis. In 
situations with insufficient pilot proficiency, relatively small investments should be spent 
on training, while a large investments should be directed to training combined with the 
best modernization program available. 

Sensitivity analysis shows that LDW methodology is less sensitive to the partial 
changes in inputs than the TASCFORM. 

The areas of application for both methodologies are approximately the same. The 
main difference lies in the level of expertise required by the methods. 

Static quality/quantity methodologies do have some limitations. For example 
TASCFORM has been designed for simplicity and ease of use. Complex processes and 
interrelationships have been represented by simplified characterizations of typical 
outcomes. The reliance on subjective inputs makes both methodologies only as good as 
the judgements that have been incorporated. Care has been taken to ensure the quality 
and objectivity of these inputs. Most importantly, static force assessment methodologies 
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are not useful in predicting combat outcomes. Their utility lies in shedding light on the 
correlation of military forces through side-by-side comparison of conventional force 
elements and associated trends. 



83 



THIS PAGE INTENT! >NALLLY LEFT BLANK 



84 



VI. RECOMMENDATIONS 



The real modernization decision-making will take place in a dynamic context 
political environment, developing of the force structure and the ongoing military force 
modernization of in all of Ukraine’s neighboring countries. This will demand careful 
analysis. A variety of flexible analytical tools must be used whose results must be 
integrated. These tools range from simple static analyses ("the bean count"), through 
more sophisticated force quantity/quality indexes, to the full range of dynamic 
techniques, such as simulations and war games. However, several cautions are in order. 
(Vogt, p. 1) 

First, no single technique or small set of techniques can provide the "correct" 
answer; each has its own advantages and drawbacks. An understanding of air force 
capabilities requires the integration of outputs from a number of indicators, as in 
economic analysis. Economic analysis is a complex field that has produced a broad range 
of static and dynamic analytical and predictive techniques. Different theoretical 
perceptions of the overall process have resulted and a substantial degree of uncertainty 
remains. The same is true of net assessment of any air forces. 

Second, any comprehensive analysis must acknowledge that the participants will 
disagree about the main features of the air forces modernization program, due to the 
political concerns and attempts to obtain an advantage by exaggerating the potential 
problems of the particular service. Thus, the same analyses need to be applied to 
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different sets of data, and the results examined to see whether the disagreements have a 
significant effect on the overall assessment of the alternatives. (Vogt, p. 2) 

Cost containment is not always synonymous with improved military capability. 
In conditions of insufficient financing achieving the highest possible benefit from the 
investment is critical for military programs. Quantification of the value of the military 
benefits of different modernization programs, is seen as an increasingly important process 
and is central to the role of the MOD Armament Department. 

Providing advice on cost-effectiveness and cost-utility analysis in the MOD 
would involve the interpretation of an economic evaluation conducted from the 
perspective of the military benefits (readiness, sustainability, etc) provided by the 
modernization program. Budgetary limits make economic evaluations more useful and 
acceptable, as often a new weapon may have higher costs while reducing expenditure in 
other sectors of defense by reducing personnel or maintenance cost. The aim of 
economic evaluation is to determine how available resources can be used most 
efficiently. All economic evaluations must satisfy two criteria to qualify as full 
economic evaluations. There must be: 

• Comparative analysis of two or more alternatives; 

• A full exploration in all cases of both the costs (and inputs) and the 
consequences (or outcomes). 

• All economic analyses should comprise the following three dimensions: 

• The type of analysis; 

• The perspective of the analysis; 
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• The type of costs that are included. 

Types of analysis. There are three main types of full economic analysis: cost- 
effectiveness analysis, cost-benefit analysis and cost-utility analysis. All show the 
economic efficiency associated with a particular course of action compared to an 
alternative course of action. All these methods measure costs in monetary terms, but they 
differ in how outcomes are expressed. 

The perspective of an analysis is an important concept and one that must be stated 
at the outset of a study. The perspective defines the range of costs and outcomes that 
must be included in the analysis. If it is not stated it is impossible to tell whether all 
relevant costs and consequences have been discussed. 

Practical hints. Always look very carefully at the literature and ask the 
following basic questions when you see a claim that a military program is “cost- 
effective”: 

• Has an economic evaluation been carried out? 

• Was there a comparison of the costs and outcomes of the military 
program? 

• Was there a comparison of at least two or more appropriate alternatives? 

• What type of analysis was used? 

• Was it appropriate? 

Finally, each individual must come to his own conclusions about how much 
credibility the results have and how strong the arguments are that have been made in the 
analysis. 
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APPENDIX A. TASCFORM-AIR MODEL METHODOLOGICAL OVERVIEW 



The TASCFORM-AIR model generates measures of effectiveness individual 
aircraft and for tactical air forces. The model includes all principal combat aircraft 
whose primary function is to deliver conventional ordinance against air and surface 
targets. It does not include reconnaissance, ECM, FAC, and AEW aircraft. Those 
aircraft are included in the TASCFORM-C3I model. The model uses two broad roles for 
analysis purposes: air combat and face attack. Within each role are a number of 
functions or missions, as shown in Table 1. 



AIR COMBAT 


SURFACE ATTACK 


Fighter 

Interceptor 


Close air support Heavy Bomber battlefield air 
Interdiction (CAS/BAI) launcher (ASM) 
Interdiction Antiship cruise Attack helicopter 
missile (ASCM) launcher 
Heavy bomber 

Heavy air- to- surface missile (ASM) launcher 



Table 1. TAC AIR Roles 

Computing the measure of effectiveness for a single aircraft is a three-step 
process. First, a weapon potential score is computed by scoring the airframe, power 
plant, and payload characteristics and normalizing that score against a baseline aircraft, 
the U.S. FEB. Subjective weighting factors assign relative importance to the 
characteristics depending on how they contribute to the air combat or surface attack 
role. In cases where a new model of an existing aircraft possesses a significant change 
in capability, that model is treated as a separate aircraft. Next a weapon system potential 
score is computed by scoring on board systems, such as weapons, navigation, and 
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avionics. Finally, an adjusted weapon system potential score is computed which scores 
relative productivity and the existence of an adaptive enemy. The adjusted weapons 
system score can be modified by a relative obsolescence factor to generate a depreciated 
score if desired. 

Computing the measure of effectiveness of air forces is a two-step process. First, 
a designated force potential score is computed. This score is the sum of scores of the 
individual aircraft performing a given role. Next, an equivalent force potential measure 
of effectiveness is computed by scoring relative obsolescence, multirole capability, and 
the effect of numbers, C3I capabilities, logistics, and personnel quality. The method of 
computing measures of effectiveness for individual aircraft and air forces is summarized 
bellow: 

WEAPON POTENTIAL (WP) - Basic system measure of effectiveness 
reflecting: 

• Payload 

• Aircraft range, basing modes, and standoff weapon range 

• Maneuverability 

• Speed 

WEAPON SYSTEM POTENTIAL (WSP) - Adjusts WP for: 

• Target acquisition and guidance/fire control 

• Susceptibility to countermeasures 

• Weapon enhancements 

• Navigation 
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• Survivability 

ADJUSTED WEAPON SYSTEM POTENTIAL (AWSP) - Includes: 

• Obsolescence* 

• Productivity 

DESIGNATED FORCE POTENTIAL (DFP) ~ Basic force level measure of 
effectiveness considers: 

• Distribution of assets to TACAIR roles 

• Inventory levels 

EQUIVALENT FORCE POTENTIAL (EFP) - Adjusts for: 

• C3I system effects 

• A aircrew proficiency 

• Logistics and maintenance 

• Multi-role capability 

• Tactical impact of inventory changes 

*Can be included to produce depreciated measures of effectiveness or excluded at the 
analyst's discretion 
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APPENDIX B. TASCFORM MIG-29 
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APPENDIX C. MULTI-ATTRIBUTE UTILITY THEORY 

Multi-Attribute Utility Theory (MAUT) applies to situation with uncertainty and 
multiple, often conflicting, objectives. For situation where the decision variables are 
independent, a weighted addition of utilities (i.e., linear) is used to produce an ordinal 
ranking of alternatives. When interactions between the variables exist, multiplicative 
terms are introduced, resulting in a multi-linear overall utility function. A general form 
of this equation is: 

n 

u(x) = YjkiU i ( <x) + 

i=i 

n n 

YsYjkij Ui(x)Uj(Xj ) + £ u^x)uM)uiXi + ■ • • + 

i-l j> 1 i=l j>i e> j 

kl2i N Ui ( x) Ul (X 2 )""Un ( A n ) 

where: 

1. n is normalized by u , . . . = 0 (the least preferred level of 



* * * 

all measures) and u (j£ , , ... j^ 3 ) =1 (the most preferred level of all measures). 



2. u. (% ) is a conditional utility function of normalized by u. (%*) = 0 

and Uj(xJ = l - 

3. The scaling constants can be evaluated by: £ p3 n = l- 



YjU (xj,) + - + 



(- 1 ) + 

ij> 1 i 




99 



THIS PAGE INTENTIONALLLY LEFT BLANK 



100 



APPENDIX D. PEREGRINE MODEL 



The basic PEREGRINE model is multiplicative. That is, if "A" is the availability 
term, "M" the mission effectiveness term, and "S" the survivability term, the overall 
system potential P=A*M*S. Consider two hypothetical aircraft. One has outstanding 
availability and mission effectiveness scores but has terrible survivability while the other 
is "good but not great" in all three areas. Suppose A, M, and S are scaled from 1 to 10. 
Our first aircraft might rate 9-9-1 while our second rates 6-6-6. An additive model would 
score the two aircraft as 19 versus 18, indicating that the first aircraft by virtue of its very 
high availability and mission effectiveness is slightly superior to the "good but not great" 
aircraft. This despite the fact that it is going to get shot out of the sky any time it faces a 
capable air defense system. The multiplicative model would score the two planes as 81 
for the first versus 216 for the second, indicating that the second aircraft is much better 
since it has no crippling weaknesses. This is far more supportable on both intuitive 
grounds and by historical examples. 

The "A", "M", and "S" factors each are composed of sub-terms and these may 
combine additively, multiplicatively, or in some other fashion depending on the 
relationship between the factors involved. For example, the ability of one aircraft to shoot 
another down will depend on (among other factors) the first aircraft’s ability to detect the 
target, to maneuver into shooting position, and to hit the target with its weapons. This is 
best expressed by a multiplicative relation between these factors since an inability to do 
any one of them will prevent the aircraft from hitting the target. But the effectiveness of 
the aircraft’s radar guided missiles (for use at long range) is independent of the 
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effectiveness of its infrared missiles (for use in a short range "dog fight"). Even if it had 
zero capability to use one of these weapons types, it might have great effectiveness with 
the other. Thus multiplying a short-range missile factor by a long-range factor to compute 
an overall value for the aircraft’s weapons suite would be inaccurate. Some other method 
of combining the factors would be more realistic. 

Though the general purpose of PEREGRINE is to provide an overall measure of 
system potential for fighter aircraft in different missions, that potential will be not just 
mission but scenario dependent. For example, if two aircraft are being compared in the 
same mission and one has adequate survivability and very high lethality while the other 
reverses those characteristics, the relative value of the two will depend greatly on enemy 
defenses. But even the "adequate" survivability might suffice if the aircraft is supported 
by advanced fighters and specialized defense suppression aircraft. No static model can 
capture all these dynamics but PEREGRINE attempts to approximate them. 
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APPENDIX E. LDW RESULTS IN EXCEL 
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APPENDIX F. GRIPPEN CONCEPT OF AIRCRAFT EFFICIENCY 
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APPENDIX G. DECISION-SUPPORT AND ACCOUNTING SOFTWARE 

EVALUATION TOOLS 

EXPERT CHOICE. Expert Choice, Inc. offers two main products for sale, 
ECPro (for use by individual decision makers) and TeamEC (a group decision-making 
tool). Both products are for Windows and are based on the analytic hierarchy process 
(AHP). The AHP is a decision tool that is for use with decisions involving multiple 
attributes or criteria and multiple alternative solutions, www.expertchoice.com 

LOGICAL DECISIONS FOR WINDOWS. Logical Decisions for Windows 
(LDW) is a decision-support software tool that uses both the AHP and other multiple 
variable decision approaches to help decision makers make choices in complex problem 
situations. Like Expert Choice, LDW evaluates multiple attributes and multiple 
alternatives. The software can be tailored to a variety of decision contexts, such as the 
accounting software selection decision, www.logicaldecisions.com 

REQUIREMENTS ANALYST. Computer Training Services, Inc., has sold 
Requirements Analyst for many years. The software is regularly updated with new data 
about accounting software products. Decision makers supply information about the 
software features they either need or would like to have, and Requirements Analyst 
produces a list of the features associated with the optimal accounting software choice. 
ww.ctsguides. com 

SOFTWARE COMPARE. The home page address for Practitioners Publishing 
Company, vendors for this product, is www.ppcinfo.com. Practitioners Publishing 
Company also sells a manual. Guide to Installing Microcomputer Accounting Systems, at 
this site. 
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THE ACCOUNTING LIBRARY. The Accounting Library (TAL) is a software 
tool designed specifically for the accounting software selection decision. Software 
vendors furnish data about their software features to the program’s developer. TAL users 
supply data about those software features that are of particular importance. The software 
includes more than 100 software programs and more than 1,500 factors or software 
features. There are several versions of the software available at www.excelco.com. 

ACCOUNTING/FINANCIAL MANAGEMENT SOFTWARE-RELATED 
INTERNET SITES 

www.accountingdirecdtory.com This site includes a directory of accounting 
software companies and other resources for information about accounting systems. From 
the site, you may access a consulting service, Accountsoft, that chooses your accounting 
software solution for a fee. 

www.acctg2000.com A website that offers assistance in selecting the best 
accounting software package for your business. The site includes a list of accounting 
software publishers’ names and sources of software support. There are links to a variety 
of other sites, such as educational/training sources, that can also provide you with help. 

www.cpaonline.com The site includes news items related to technology that are 
likely to be of interest to accountants. The site also has links to a seminar series offering 
guidance in accounting software selection (Business and Accounting Software Update), a 
summary of accounting software packages, and a list of accounting software consultants 
in your business location. 

www.ctsguides.com This is the home page for Computer Training Services, a 
Rockville, Md., company that sells products and services for consultants and managers to 
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use in selecting accounting software. The company sells software guides for special 
industries in addition to their premier product. The Requirements Analyst. The site also 
includes advice about the software selection process from needs analysis to 
implementation. 

www.excelco.com The site is an accounting system selector that advertises The 
Accounting Library, You can buy all versions of The Accounting Library at this site or 
purchase consulting services that rely on it. 

www.fsforum.com This site is a not-for-profit site where financial systems 
professionals exchange information in discussion groups related to their interests. The 
site includes resources such as a free software search service, vendor links, a resource 
library with articles about financial systems, and Y2K resources. 

www.k2e.com/ac/accountingsoftware.htm A comprehensive Web source with 
information about accounting software for accountants and consultants. The site, 
developed by K2 Enterprises, a company specializing in offering accounting system 
selection support, includes information on accounting software packages, software 
evaluations, links to software vendors, manual and book resources available, and 
information on software selection seminars. 

www.mamag.com/strategicfinance/vendors.htm The online listing of software 
vendors from Strategic Finance. 



www.softwarenews.net/buyers/account.htm A software news service for 
accounting software buyers containing links to software vendors. 
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